From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A52DFA0093 for ; Fri, 11 Mar 2022 08:18:28 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 584D94113F; Fri, 11 Mar 2022 08:18:28 +0100 (CET) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id 4ABAC40042 for ; Fri, 11 Mar 2022 08:18:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1646983107; x=1678519107; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=IlZzlpZUYF1bwgWsGW4z4r/tr/Je6vNOK8Vc+/DUslA=; b=V+KA3tAS4U79qpFG+Jwn0nQgs8yEdDXzGYWuiDunwh13d2iJSXQX/Bb5 N0GpT/0Awh7cG+8zWDNGeqZMsIX762aIhHYINbmac9hdIvaKMK2+1ITG+ LIEj+DEBiBxhwT6xa7U8cLmbZHYZ6Bmh7RFmAKXYfcYXDLeEhG37YBRiK L/B6pfPM9LYmG75GYAk+8ThDwT4b3LhscW2XkbHhYbO6cJ8U5Dr57EYIK Ecm9psLVGTZR6nAd82gLVDfBvglnkLKl2e87iV6VTZWEcDGBKJlbOgnvu CmqABTqcCkoS3ZdoXvQwWYmH4MM19Js9HhnGYaaXidj7VYFg/DRATIAzk A==; X-IronPort-AV: E=McAfee;i="6200,9189,10282"; a="318741584" X-IronPort-AV: E=Sophos;i="5.90,173,1643702400"; d="scan'208";a="318741584" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Mar 2022 23:18:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,173,1643702400"; d="scan'208";a="633325975" Received: from unknown (HELO localhost.localdomain.sh.intel.com) ([10.238.175.107]) by FMSMGA003.fm.intel.com with ESMTP; 10 Mar 2022 23:18:25 -0800 From: Wei Huang To: stable@dpdk.org, christian.ehrhardt@canonical.com Cc: rosen.xu@intel.com, tianfei.zhang@intel.com, Wei Huang Subject: [PATCH 19.11 2/2] raw/ifpga: fix monitor thread Date: Fri, 11 Mar 2022 02:15:05 -0500 Message-Id: <20220311071505.1507945-3-wei.huang@intel.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20220311071505.1507945-1-wei.huang@intel.com> References: <20220311071505.1507945-1-wei.huang@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org [ upstream commit 2479a1e9a8886c93ba4c8a7aa2a35a04302f1aae ] Monitor thread handles graceful shutdown according to the value of specific sensors in device, two issues are found below. 1. Thread is not created when card is probed. 2. Thread is canceled without checking presence of other cards. To fix them, thread is created in pci device probe function, a reference count is checked before canceling the thread. Fixes: 9c006c4 ("raw/ifpga: scan PCIe BDF device tree") Cc: stable@dpdk.org Signed-off-by: Wei Huang Acked-by: Tianfei Zhang --- drivers/raw/ifpga/ifpga_rawdev.c | 80 ++++++++++++++++++++++------------------ drivers/raw/ifpga/ifpga_rawdev.h | 2 + 2 files changed, 46 insertions(+), 36 deletions(-) diff --git a/drivers/raw/ifpga/ifpga_rawdev.c b/drivers/raw/ifpga/ifpga_rawdev.c index dfebb7e..b385e17 100644 --- a/drivers/raw/ifpga/ifpga_rawdev.c +++ b/drivers/raw/ifpga/ifpga_rawdev.c @@ -76,7 +76,7 @@ static struct ifpga_rawdev ifpga_rawdevices[IFPGA_RAWDEV_NUM]; -static int ifpga_monitor_start; +static int ifpga_monitor_refcnt; static pthread_t ifpga_monitor_start_thread; static struct ifpga_rawdev * @@ -141,6 +141,7 @@ struct ifpga_rawdev * dev->dev_id = dev_id; for (i = 0; i < IFPGA_MAX_IRQ; i++) dev->intr_handle[i] = NULL; + dev->poll_enabled = 0; return dev; } @@ -214,10 +215,10 @@ static int ifpga_get_dev_vendor_id(const char *bdf, return 0; } -static int ifpga_rawdev_fill_info(struct ifpga_rawdev *ifpga_dev, - const char *bdf) +static int ifpga_rawdev_fill_info(struct ifpga_rawdev *ifpga_dev) { - char path[1024] = "/sys/bus/pci/devices/0000:"; + struct opae_adapter *adapter = NULL; + char path[1024] = "/sys/bus/pci/devices/"; char link[1024], link1[1024]; char dir[1024] = "/sys/devices/"; char *c; @@ -232,7 +233,11 @@ static int ifpga_rawdev_fill_info(struct ifpga_rawdev *ifpga_dev, int func; uint32_t dev_id, vendor_id; - strlcat(path, bdf, sizeof(path)); + adapter = ifpga_dev ? ifpga_rawdev_get_priv(ifpga_dev->rawdev) : NULL; + if (!adapter) + return -ENODEV; + + strlcat(path, adapter->name, sizeof(path)); memset(link, 0, sizeof(link)); memset(link1, 0, sizeof(link1)); ret = readlink(path, link, (sizeof(link)-1)); @@ -382,7 +387,7 @@ static int ifpga_rawdev_fill_info(struct ifpga_rawdev *ifpga_dev, /* monitor temperature sensors */ if (!strcmp(sensor->name, "Board Temperature") || !strcmp(sensor->name, "FPGA Die Temperature")) { - IFPGA_RAWDEV_PMD_INFO("read sensor %s %d %d %d\n", + IFPGA_RAWDEV_PMD_DEBUG("read sensor %s %d %d %d\n", sensor->name, value, sensor->high_warn, sensor->high_fatal); @@ -424,7 +429,7 @@ static int set_surprise_link_check_aer( bool enable = 0; uint32_t aer_new0, aer_new1; - if (!ifpga_rdev) { + if (!ifpga_rdev || !ifpga_rdev->rawdev) { printf("\n device does not exist\n"); return -EFAULT; } @@ -503,11 +508,11 @@ static int set_surprise_link_check_aer( int gsd_enable, ret; #define MS 1000 - while (1) { + while (ifpga_monitor_refcnt) { gsd_enable = 0; for (i = 0; i < IFPGA_RAWDEV_NUM; i++) { ifpga_rdev = &ifpga_rawdevices[i]; - if (ifpga_rdev->rawdev) { + if (ifpga_rdev->poll_enabled) { ret = set_surprise_link_check_aer(ifpga_rdev, gsd_enable); if (ret == 1 && !gsd_enable) { @@ -527,30 +532,45 @@ static int set_surprise_link_check_aer( } static int -ifpga_monitor_start_func(void) +ifpga_monitor_start_func(struct ifpga_rawdev *dev) { int ret; - if (ifpga_monitor_start == 0) { - ret = pthread_create(&ifpga_monitor_start_thread, - NULL, + if (!dev) + return -ENODEV; + + ret = ifpga_rawdev_fill_info(dev); + if (ret) + return ret; + + dev->poll_enabled = 1; + + if (ifpga_monitor_refcnt++ == 0) { + ret = rte_ctrl_thread_create(&ifpga_monitor_start_thread, + "ifpga-monitor", NULL, ifpga_rawdev_gsd_handle, NULL); if (ret) { + ifpga_monitor_start_thread = 0; IFPGA_RAWDEV_PMD_ERR( - "Fail to create ifpga nonitor thread"); + "Fail to create ifpga monitor thread"); return -1; } - ifpga_monitor_start = 1; } return 0; } static int -ifpga_monitor_stop_func(void) +ifpga_monitor_stop_func(struct ifpga_rawdev *dev) { int ret; - if (ifpga_monitor_start == 1) { + if (!dev || !dev->poll_enabled) + return 0; + + dev->poll_enabled = 0; + + if ((--ifpga_monitor_refcnt == 0) && + ifpga_monitor_start_thread) { ret = pthread_cancel(ifpga_monitor_start_thread); if (ret) IFPGA_RAWDEV_PMD_ERR("Can't cancel the thread"); @@ -559,8 +579,6 @@ static int set_surprise_link_check_aer( if (ret) IFPGA_RAWDEV_PMD_ERR("Can't join the thread"); - ifpga_monitor_start = 0; - return ret; } @@ -1550,6 +1568,10 @@ static int fme_clean_fme_error(struct opae_manager *mgr) if (ret) goto free_adapter_data; + ret = ifpga_monitor_start_func(dev); + if (ret) + goto free_adapter_data; + return ret; free_adapter_data: @@ -1592,6 +1614,8 @@ static int fme_clean_fme_error(struct opae_manager *mgr) } dev = ifpga_rawdev_get(rawdev); + ifpga_monitor_stop_func(dev); + adapter = ifpga_rawdev_get_priv(rawdev); if (!adapter) return -ENODEV; @@ -1626,7 +1650,7 @@ static int fme_clean_fme_error(struct opae_manager *mgr) static int ifpga_rawdev_pci_remove(struct rte_pci_device *pci_dev) { - ifpga_monitor_stop_func(); + IFPGA_RAWDEV_PMD_INFO("remove pci_dev %s", pci_dev->device.name); return ifpga_rawdev_destroy(pci_dev); } @@ -1679,11 +1703,8 @@ static int ifpga_rawdev_get_string_arg(const char *key __rte_unused, { struct rte_devargs *devargs; struct rte_kvargs *kvlist = NULL; - struct rte_rawdev *rawdev = NULL; - struct ifpga_rawdev *ifpga_dev; int port; char *name = NULL; - const char *bdf; char dev_name[RTE_RAWDEV_NAME_MAX_LEN]; int ret = -1; @@ -1725,19 +1746,6 @@ static int ifpga_rawdev_get_string_arg(const char *key __rte_unused, } memset(dev_name, 0, sizeof(dev_name)); - snprintf(dev_name, RTE_RAWDEV_NAME_MAX_LEN, "IFPGA:%s", name); - rawdev = rte_rawdev_pmd_get_named_dev(dev_name); - if (!rawdev) - goto end; - ifpga_dev = ifpga_rawdev_get(rawdev); - if (!ifpga_dev) - goto end; - bdf = name; - ifpga_rawdev_fill_info(ifpga_dev, bdf); - - ifpga_monitor_start_func(); - - memset(dev_name, 0, sizeof(dev_name)); snprintf(dev_name, RTE_RAWDEV_NAME_MAX_LEN, "%d|%s", port, name); diff --git a/drivers/raw/ifpga/ifpga_rawdev.h b/drivers/raw/ifpga/ifpga_rawdev.h index d82b887..7be4133 100644 --- a/drivers/raw/ifpga/ifpga_rawdev.h +++ b/drivers/raw/ifpga/ifpga_rawdev.h @@ -60,6 +60,8 @@ struct ifpga_rawdev { char parent_bdf[16]; /* 0 for FME interrupt, others are reserved for AFU irq */ void *intr_handle[IFPGA_MAX_IRQ]; + /* enable monitor thread poll device's sensors or not */ + int poll_enabled; }; struct ifpga_rawdev * -- 1.8.3.1