From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <tiwei.bie@intel.com>
Received: from mga18.intel.com (mga18.intel.com [134.134.136.126])
by dpdk.org (Postfix) with ESMTP id 3381C28FD;
Wed, 5 Sep 2018 06:31:02 +0200 (CEST)
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from fmsmga008.fm.intel.com ([10.253.24.58])
by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
04 Sep 2018 21:31:00 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.53,332,1531810800"; d="scan'208";a="68456347"
Received: from btwcube1.sh.intel.com ([10.67.104.194])
by fmsmga008.fm.intel.com with ESMTP; 04 Sep 2018 21:29:47 -0700
From: Tiwei Bie <tiwei.bie@intel.com>
To: maxime.coquelin@redhat.com,
zhihong.wang@intel.com,
dev@dpdk.org
Cc: seanbh@gmail.com,
anatoly.burakov@intel.com,
stable@dpdk.org
Date: Wed, 5 Sep 2018 12:28:50 +0800
Message-Id: <20180905042852.6212-2-tiwei.bie@intel.com>
X-Mailer: git-send-email 2.18.0
In-Reply-To: <20180905042852.6212-1-tiwei.bie@intel.com>
References: <20180905042852.6212-1-tiwei.bie@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Subject: [dpdk-dev] [PATCH 1/3] net/virtio-user: fix deadlock in memory
events callback
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Wed, 05 Sep 2018 04:31:02 -0000
Deadlock can occur when allocating memory if a vhost-kernel
based virtio-user device is in use. To fix the deadlock,
we will take memory hotplug lock explicitly in virtio-user
when necessary, and always call the _thread_unsafe memory
functions.
Bugzilla ID: 81
Fixes: 12ecb2f63b12 ("net/virtio-user: support memory hotplug")
Cc: stable@dpdk.org
Reported-by: Seán Harte <seanbh@gmail.com>
Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
---
drivers/net/virtio/virtio_user/vhost_kernel.c | 6 +++++-
.../net/virtio/virtio_user/virtio_user_dev.c | 19 +++++++++++++++++++
2 files changed, 24 insertions(+), 1 deletion(-)
diff --git a/drivers/net/virtio/virtio_user/vhost_kernel.c b/drivers/net/virtio/virtio_user/vhost_kernel.c
index b2444096c..897fee0af 100644
--- a/drivers/net/virtio/virtio_user/vhost_kernel.c
+++ b/drivers/net/virtio/virtio_user/vhost_kernel.c
@@ -115,7 +115,11 @@ prepare_vhost_memory_kernel(void)
wa.region_nr = 0;
wa.vm = vm;
- if (rte_memseg_contig_walk(add_memory_region, &wa) < 0) {
+ /*
+ * The memory lock has already been taken by memory subsystem
+ * or virtio_user_start_device().
+ */
+ if (rte_memseg_contig_walk_thread_unsafe(add_memory_region, &wa) < 0) {
free(vm);
return NULL;
}
diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c b/drivers/net/virtio/virtio_user/virtio_user_dev.c
index 7df600b02..869e96f87 100644
--- a/drivers/net/virtio/virtio_user/virtio_user_dev.c
+++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c
@@ -13,6 +13,8 @@
#include <sys/types.h>
#include <sys/stat.h>
+#include <rte_eal_memconfig.h>
+
#include "vhost.h"
#include "virtio_user_dev.h"
#include "../virtio_ethdev.h"
@@ -109,9 +111,24 @@ is_vhost_user_by_type(const char *path)
int
virtio_user_start_device(struct virtio_user_dev *dev)
{
+ struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config;
uint64_t features;
int ret;
+ /*
+ * XXX workaround!
+ *
+ * We need to make sure that the locks will be
+ * taken in the correct order to avoid deadlocks.
+ *
+ * Before releasing this lock, this thread should
+ * not trigger any memory hotplug events.
+ *
+ * This is a temporary workaround, and should be
+ * replaced when we get proper supports from the
+ * memory subsystem in the future.
+ */
+ rte_rwlock_read_lock(&mcfg->memory_hotplug_lock);
pthread_mutex_lock(&dev->mutex);
if (is_vhost_user_by_type(dev->path) && dev->vhostfd < 0)
@@ -152,10 +169,12 @@ virtio_user_start_device(struct virtio_user_dev *dev)
dev->started = true;
pthread_mutex_unlock(&dev->mutex);
+ rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock);
return 0;
error:
pthread_mutex_unlock(&dev->mutex);
+ rte_rwlock_read_unlock(&mcfg->memory_hotplug_lock);
/* TODO: free resource here or caller to check */
return -1;
}
--
2.18.0