From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BA0B4A04FD for ; Tue, 12 Jul 2022 21:23:58 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B182B40F1A; Tue, 12 Jul 2022 21:23:58 +0200 (CEST) Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) by mails.dpdk.org (Postfix) with ESMTP id BED8F427EE for ; Tue, 12 Jul 2022 21:23:55 +0200 (CEST) Received: by mail-wm1-f52.google.com with SMTP id i128-20020a1c3b86000000b003a2ce31b4f8so7324125wma.1 for ; Tue, 12 Jul 2022 12:23:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=K/Piy7fN5wMrdwW0eYzCnbTuXnDXfcscLoDEPbi6hH4=; b=hHxDtg8znIYgIqmn8bJ0ZrP0cgHU32CXi4zAVn1stPQJnrx+Ejj2Huw0+Q958zQ6jC rOCHc/N4V9M5fZj3An7CLcRylVH8LM4yBuy3NMe4UCfaw6KYRFl4H+df+crJtIXzdmmS 33aWVn5kmX07e7UfBHlwQ00AS78BtxErZ3kHitJXFJYbV1tMOtgLGxCi2XbIAndJE2K4 cnp4TwomLpZZTW66mrFdUHoLpWnZNTR6p96sPx/9PyO2SLcXN5fY9gO3zW2L8b+rJ5lT 6ukgjTost7ScsDQ1VRX6fTtJ6AIfKv0DphugVoqDgO1W66wQgU8d36YMwm11lB7VguGy V/UA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=K/Piy7fN5wMrdwW0eYzCnbTuXnDXfcscLoDEPbi6hH4=; b=UFxSmDaHtrppSRBDPUN2E5KD4Zs2SYQ/KS5Rq7GWtgXaprODK+ijBYvDPAkhY6pDKS nTE/2/76LeTiSdbiBokrhx0KRHPVPpi1Q2jQDPpy2gMONxH90t0/mOBDaJZ5nMzX9TS1 Bcy4ERoX+jMaTwtpatEK72Wqg2gR03/gdNv6L7Y5hEdyhlIwQA/Yhtt3D5jX9/PxwVqB RSYMM0hftywRCTuTpSsFmpQmlddkTCcrtqwVGUXBURl4ltUl+9o6fb2udH/H8dx0i22h URvFzrV+873lekgFJRQi8d2nHaW6d+sdiotsgH62kAnV9nnPPB/gIflVs3FXt2qzdn8G NG3A== X-Gm-Message-State: AJIora9zRYfUl7npdniLClwE1Pj9IuB1r8/v7LcXZSGwzv0tBETiA3k7 brYGbgxYJP29syJxMI8V6Nc= X-Google-Smtp-Source: AGRyM1uYiKMCs9MM2tzKsGENYbF1Kg6UoQSPZZXufi97L500vZcLNRQ19sUw9j8j4RYxgHr5gBrDVQ== X-Received: by 2002:a05:600c:294:b0:3a2:cf18:cc8d with SMTP id 20-20020a05600c029400b003a2cf18cc8dmr5465325wmk.92.1657653835421; Tue, 12 Jul 2022 12:23:55 -0700 (PDT) Received: from localhost ([2a01:4b00:f41a:3600:360b:9754:2e3a:c344]) by smtp.gmail.com with ESMTPSA id w13-20020adff9cd000000b0021d20461bbbsm8959166wrr.88.2022.07.12.12.23.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Jul 2022 12:23:54 -0700 (PDT) From: luca.boccassi@gmail.com To: Abhimanyu Saini Cc: Maxime Coquelin , dpdk stable Subject: patch 'vdpa/sfc: resolve race between vhost lib and device conf' has been queued to stable release 21.11.2 Date: Tue, 12 Jul 2022 20:23:47 +0100 Message-Id: <20220712192349.4174428-4-luca.boccassi@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220712192349.4174428-1-luca.boccassi@gmail.com> References: <20220706203506.3422496-31-luca.boccassi@gmail.com> <20220712192349.4174428-1-luca.boccassi@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Hi, FYI, your patch has been queued to stable release 21.11.2 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 07/14/22. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://github.com/kevintraynor/dpdk-stable This queued commit can be viewed at: https://github.com/kevintraynor/dpdk-stable/commit/b11e955370a713ee418ee0f5e853364fe72132a8 Thanks. Luca Boccassi --- >From b11e955370a713ee418ee0f5e853364fe72132a8 Mon Sep 17 00:00:00 2001 From: Abhimanyu Saini Date: Wed, 6 Jul 2022 14:54:01 +0530 Subject: [PATCH] vdpa/sfc: resolve race between vhost lib and device conf [ upstream commit d9a1ef20f0f1c6f4029f79d5eab5dc696d115c81 ] libvhost calls dev_conf() before prosessing the VHOST_USER_SET_VRING_CALL message for the last VQ. So this message is processed after dev_conf() returns. However, the dev_conf() function spawns a thread to set rte_vhost_host_notifier_ctrl() before returning control to libvhost. This parallel thread in turn invokes get_notify_area(). To get the notify_area, the vdpa driver needs to query the HW and for this query it needs an enabled VQ. But at the same time libvhost is processing the last VHOST_USER_SET_VRING_CALL, and to do that it disables the last VQ. Hence there is a race b/w the libvhost and the vdpa driver. To resolve this race condition, query the HW and cache notify_area inside dev_conf() instead of doing it the parallel thread. Fixes: 630be406dcbf ("vdpa/sfc: get queue notify area info") Signed-off-by: Abhimanyu Saini Reviewed-by: Maxime Coquelin --- drivers/vdpa/sfc/sfc_vdpa_ops.c | 36 ++++++++++++++++----------------- drivers/vdpa/sfc/sfc_vdpa_ops.h | 1 + 2 files changed, 19 insertions(+), 18 deletions(-) diff --git a/drivers/vdpa/sfc/sfc_vdpa_ops.c b/drivers/vdpa/sfc/sfc_vdpa_ops.c index 63aa52da49..b84699d234 100644 --- a/drivers/vdpa/sfc/sfc_vdpa_ops.c +++ b/drivers/vdpa/sfc/sfc_vdpa_ops.c @@ -222,6 +222,7 @@ static int sfc_vdpa_virtq_start(struct sfc_vdpa_ops_data *ops_data, int vq_num) { int rc; + uint32_t doorbell; efx_virtio_vq_t *vq; struct sfc_vdpa_vring_info vring; efx_virtio_vq_cfg_t vq_cfg; @@ -270,22 +271,35 @@ sfc_vdpa_virtq_start(struct sfc_vdpa_ops_data *ops_data, int vq_num) /* Start virtqueue */ rc = efx_virtio_qstart(vq, &vq_cfg, &vq_dyncfg); if (rc != 0) { - /* destroy virtqueue */ sfc_vdpa_err(ops_data->dev_handle, "virtqueue start failed: %s", rte_strerror(rc)); - efx_virtio_qdestroy(vq); goto fail_virtio_qstart; } sfc_vdpa_info(ops_data->dev_handle, "virtqueue started successfully for vq_num %d", vq_num); + rc = efx_virtio_get_doorbell_offset(vq, &doorbell); + if (rc != 0) { + sfc_vdpa_err(ops_data->dev_handle, + "failed to get doorbell offset: %s", + rte_strerror(rc)); + goto fail_doorbell; + } + + /* + * Cache the bar_offset here for each VQ here, it will come + * in handy when sfc_vdpa_get_notify_area() is invoked. + */ + ops_data->vq_cxt[vq_num].doorbell = (void *)(uintptr_t)doorbell; ops_data->vq_cxt[vq_num].enable = B_TRUE; return rc; +fail_doorbell: fail_virtio_qstart: + efx_virtio_qdestroy(vq); fail_vring_info: return rc; } @@ -792,8 +806,6 @@ sfc_vdpa_get_notify_area(int vid, int qid, uint64_t *offset, uint64_t *size) int ret; efx_nic_t *nic; int vfio_dev_fd; - efx_rc_t rc; - unsigned int bar_offset; volatile void *doorbell; struct rte_pci_device *pci_dev; struct rte_vdpa_device *vdpa_dev; @@ -824,19 +836,6 @@ sfc_vdpa_get_notify_area(int vid, int qid, uint64_t *offset, uint64_t *size) return -1; } - if (ops_data->vq_cxt[qid].enable != B_TRUE) { - sfc_vdpa_err(dev, "vq is not enabled"); - return -1; - } - - rc = efx_virtio_get_doorbell_offset(ops_data->vq_cxt[qid].vq, - &bar_offset); - if (rc != 0) { - sfc_vdpa_err(dev, "failed to get doorbell offset: %s", - rte_strerror(rc)); - return rc; - } - reg.index = sfc_vdpa_adapter_by_dev_handle(dev)->mem_bar.esb_rid; ret = ioctl(vfio_dev_fd, VFIO_DEVICE_GET_REGION_INFO, ®); if (ret != 0) { @@ -845,7 +844,8 @@ sfc_vdpa_get_notify_area(int vid, int qid, uint64_t *offset, uint64_t *size) return ret; } - *offset = reg.offset + bar_offset; + /* Use bar_offset that was cached during sfc_vdpa_virtq_start() */ + *offset = reg.offset + (uint64_t)ops_data->vq_cxt[qid].doorbell; len = (1U << encp->enc_vi_window_shift) / 2; if (len >= sysconf(_SC_PAGESIZE)) { diff --git a/drivers/vdpa/sfc/sfc_vdpa_ops.h b/drivers/vdpa/sfc/sfc_vdpa_ops.h index 6d790fd8be..9dbd5b84dd 100644 --- a/drivers/vdpa/sfc/sfc_vdpa_ops.h +++ b/drivers/vdpa/sfc/sfc_vdpa_ops.h @@ -35,6 +35,7 @@ struct sfc_vdpa_vring_info { }; typedef struct sfc_vdpa_vq_context_s { + volatile void *doorbell; uint8_t enable; uint32_t pidx; uint32_t cidx; -- 2.34.1 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2022-07-12 20:22:33.849529911 +0100 +++ 0004-vdpa-sfc-resolve-race-between-vhost-lib-and-device-c.patch 2022-07-12 20:22:33.713247244 +0100 @@ -1 +1 @@ -From d9a1ef20f0f1c6f4029f79d5eab5dc696d115c81 Mon Sep 17 00:00:00 2001 +From b11e955370a713ee418ee0f5e853364fe72132a8 Mon Sep 17 00:00:00 2001 @@ -5,0 +6,2 @@ +[ upstream commit d9a1ef20f0f1c6f4029f79d5eab5dc696d115c81 ] + @@ -25 +26,0 @@ -Cc: stable@dpdk.org