From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5017DA0093 for ; Wed, 9 Mar 2022 17:32:05 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 475E14013F; Wed, 9 Mar 2022 17:32:05 +0100 (CET) Received: from mail-wr1-f50.google.com (mail-wr1-f50.google.com [209.85.221.50]) by mails.dpdk.org (Postfix) with ESMTP id EC0724013F for ; Wed, 9 Mar 2022 17:32:04 +0100 (CET) Received: by mail-wr1-f50.google.com with SMTP id h15so3913381wrc.6 for ; Wed, 09 Mar 2022 08:32:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KIALIhaRszsb7Y3UG/gukBWm7f4KcJ2GFrmlGb3JvZI=; b=nY/UPQXiFA/ZxP+2aKuEOSJjIW1ezeMDI+lkrJjD+80MhgXSI1C2a05qDykLDC9Tyt boraTpfaoXpKnjt+9xSeGJQohDoP4EmDZCrjGH8hz+gbOEdS8Jss1bfo/+Tp+TqYBrbJ PbfT+5dm7MrxxnGnLP4Ix7wNjqOu7i9NZK3zDoFkDERb05ofEvpZiMxJXbO6BYau8idK kfUVNfKHCKAEF79XCcmijK7Xg3+rjo6ZXXn1zQ5geW2lmShjDy6z39JtK49yaXl2GYrw iSWLzsO1unPzBqEg+jmnWVlzhJCfyzjclkPd1Qfbsr83Oz1j0F3AfIt19NGU9aR/Y1WB ErAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KIALIhaRszsb7Y3UG/gukBWm7f4KcJ2GFrmlGb3JvZI=; b=VMhJXeFLQ64u4qV7M1nsN7o8SUCHLgF/4Eo+NcKtfIBCdKf9MJ7cLRpt0jhZfvyDEu s2M/NqznHP8ZXSR/z7nvV98LKfEQbYzkw8Q4tO3TB+Y2TUD2XT4HKb2R1mcpJ7/021xH 4a4PqDQLq6KcttY+TdKqLlkT1z0ll5AEin8W15AB6XbJNYlrzsKiLQxdJF2LCwPy3/Hf u7Cx/1bYd0V+Q5FcZ6afboZYeBxa84xrxuyINXjizk9rz0Lm4sCmV4/2Nx1aJiSQN3sN Vye898Ddj/NQ1Tr9pTFXSZ8uxqiOdDqvu2Bqaavui/Bno+9aSAe/ItpAgHcwGnBkwOgJ KNKg== X-Gm-Message-State: AOAM530bYa08NLlkaeWbcwFhGiQHcInvmFZQDJAWi2wYGXncMBf93qGV Vt/nwtORpCL4EPJJ790DUCdS8dcq2EE= X-Google-Smtp-Source: ABdhPJwZGYw27jbVIdRnAxdAGtq1iISc82pQVd4sx/AMSq502RuI7/mfMtbaK4GJPMjJjXLC/E9Fbw== X-Received: by 2002:a5d:6c68:0:b0:203:691f:b95b with SMTP id r8-20020a5d6c68000000b00203691fb95bmr331141wrz.375.1646843524638; Wed, 09 Mar 2022 08:32:04 -0800 (PST) Received: from localhost ([137.220.125.106]) by smtp.gmail.com with ESMTPSA id c124-20020a1c3582000000b00384d42a9638sm2268183wma.2.2022.03.09.08.32.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Mar 2022 08:32:04 -0800 (PST) From: luca.boccassi@gmail.com To: Wei Huang Cc: Tianfei Zhang , dpdk stable Subject: patch 'raw/ifpga: fix monitor thread' has been queued to stable release 20.11.5 Date: Wed, 9 Mar 2022 16:30:37 +0000 Message-Id: <20220309163053.245754-21-luca.boccassi@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220309163053.245754-1-luca.boccassi@gmail.com> References: <20220218123931.1749595-1-luca.boccassi@gmail.com> <20220309163053.245754-1-luca.boccassi@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Hi, FYI, your patch has been queued to stable release 20.11.5 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 03/11/22. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://github.com/bluca/dpdk-stable This queued commit can be viewed at: https://github.com/bluca/dpdk-stable/commit/4297b2d419135300f90ce52328d384de484c9d67 Thanks. Luca Boccassi --- >From 4297b2d419135300f90ce52328d384de484c9d67 Mon Sep 17 00:00:00 2001 From: Wei Huang Date: Tue, 1 Mar 2022 03:47:03 -0500 Subject: [PATCH] raw/ifpga: fix monitor thread [ upstream commit 2479a1e9a8886c93ba4c8a7aa2a35a04302f1aae ] Monitor thread handles graceful shutdown according to the value of specific sensors in device, two issues are found below. 1. Thread is not created when card is probed. 2. Thread is canceled without checking presence of other cards. To fix them, thread is created in pci device probe function, a reference count is checked before canceling the thread. Fixes: 9c006c45 ("raw/ifpga: scan PCIe BDF device tree") Signed-off-by: Wei Huang Acked-by: Tianfei Zhang --- drivers/raw/ifpga/ifpga_rawdev.c | 62 +++++++++++++++++++++----------- drivers/raw/ifpga/ifpga_rawdev.h | 2 ++ 2 files changed, 43 insertions(+), 21 deletions(-) diff --git a/drivers/raw/ifpga/ifpga_rawdev.c b/drivers/raw/ifpga/ifpga_rawdev.c index decf183a0b..fb37313784 100644 --- a/drivers/raw/ifpga/ifpga_rawdev.c +++ b/drivers/raw/ifpga/ifpga_rawdev.c @@ -68,7 +68,7 @@ static const struct rte_pci_id pci_ifpga_map[] = { static struct ifpga_rawdev ifpga_rawdevices[IFPGA_RAWDEV_NUM]; -static int ifpga_monitor_start; +static int ifpga_monitor_refcnt; static pthread_t ifpga_monitor_start_thread; #define IFPGA_MAX_IRQ 12 @@ -134,6 +134,7 @@ ifpga_rawdev_allocate(struct rte_rawdev *rawdev) dev = &ifpga_rawdevices[dev_id]; dev->rawdev = rawdev; dev->dev_id = dev_id; + dev->poll_enabled = 0; return dev; } @@ -208,10 +209,11 @@ static int ifpga_get_dev_vendor_id(const char *bdf, return 0; } -static int ifpga_rawdev_fill_info(struct ifpga_rawdev *ifpga_dev, - const char *bdf) + +static int ifpga_rawdev_fill_info(struct ifpga_rawdev *ifpga_dev) { - char path[1024] = "/sys/bus/pci/devices/0000:"; + struct opae_adapter *adapter = NULL; + char path[1024] = "/sys/bus/pci/devices/"; char link[1024], link1[1024]; char dir[1024] = "/sys/devices/"; char *c; @@ -226,7 +228,11 @@ static int ifpga_rawdev_fill_info(struct ifpga_rawdev *ifpga_dev, int func; uint32_t dev_id, vendor_id; - strlcat(path, bdf, sizeof(path)); + adapter = ifpga_dev ? ifpga_rawdev_get_priv(ifpga_dev->rawdev) : NULL; + if (!adapter) + return -ENODEV; + + strlcat(path, adapter->name, sizeof(path)); memset(link, 0, sizeof(link)); memset(link1, 0, sizeof(link1)); ret = readlink(path, link, (sizeof(link)-1)); @@ -376,7 +382,7 @@ ifpga_monitor_sensor(struct rte_rawdev *raw_dev, /* monitor temperature sensors */ if (!strcmp(sensor->name, "Board Temperature") || !strcmp(sensor->name, "FPGA Die Temperature")) { - IFPGA_RAWDEV_PMD_INFO("read sensor %s %d %d %d\n", + IFPGA_RAWDEV_PMD_DEBUG("read sensor %s %d %d %d\n", sensor->name, value, sensor->high_warn, sensor->high_fatal); @@ -418,7 +424,7 @@ static int set_surprise_link_check_aer( bool enable = 0; uint32_t aer_new0, aer_new1; - if (!ifpga_rdev) { + if (!ifpga_rdev || !ifpga_rdev->rawdev) { printf("\n device does not exist\n"); return -EFAULT; } @@ -497,11 +503,11 @@ ifpga_rawdev_gsd_handle(__rte_unused void *param) int gsd_enable, ret; #define MS 1000 - while (__atomic_load_n(&ifpga_monitor_start, __ATOMIC_RELAXED)) { + while (__atomic_load_n(&ifpga_monitor_refcnt, __ATOMIC_RELAXED)) { gsd_enable = 0; for (i = 0; i < IFPGA_RAWDEV_NUM; i++) { ifpga_rdev = &ifpga_rawdevices[i]; - if (ifpga_rdev->rawdev) { + if (ifpga_rdev->poll_enabled) { ret = set_surprise_link_check_aer(ifpga_rdev, gsd_enable); if (ret == 1 && !gsd_enable) { @@ -521,32 +527,46 @@ ifpga_rawdev_gsd_handle(__rte_unused void *param) } static int -ifpga_monitor_start_func(void) +ifpga_monitor_start_func(struct ifpga_rawdev *dev) { int ret; - if (!__atomic_load_n(&ifpga_monitor_start, __ATOMIC_RELAXED)) { + if (!dev) + return -ENODEV; + + ret = ifpga_rawdev_fill_info(dev); + if (ret) + return ret; + + dev->poll_enabled = 1; + + if (!__atomic_fetch_add(&ifpga_monitor_refcnt, 1, __ATOMIC_RELAXED)) { ret = pthread_create(&ifpga_monitor_start_thread, NULL, ifpga_rawdev_gsd_handle, NULL); if (ret) { + ifpga_monitor_start_thread = 0; IFPGA_RAWDEV_PMD_ERR( "Fail to create ifpga monitor thread"); return -1; } - __atomic_store_n(&ifpga_monitor_start, 1, __ATOMIC_RELAXED); } return 0; } + static int -ifpga_monitor_stop_func(void) +ifpga_monitor_stop_func(struct ifpga_rawdev *dev) { int ret; - if (__atomic_load_n(&ifpga_monitor_start, __ATOMIC_RELAXED)) { - __atomic_store_n(&ifpga_monitor_start, 0, __ATOMIC_RELAXED); + if (!dev || !dev->poll_enabled) + return 0; + dev->poll_enabled = 0; + + if (!__atomic_sub_fetch(&ifpga_monitor_refcnt, 1, __ATOMIC_RELAXED) && + ifpga_monitor_start_thread) { ret = pthread_cancel(ifpga_monitor_start_thread); if (ret) IFPGA_RAWDEV_PMD_ERR("Can't cancel the thread"); @@ -719,6 +739,7 @@ ifpga_rawdev_close(struct rte_rawdev *dev) struct opae_adapter *adapter; if (dev) { + ifpga_monitor_stop_func(ifpga_rawdev_get(dev)); adapter = ifpga_rawdev_get_priv(dev); if (adapter) { opae_adapter_destroy(adapter); @@ -1522,6 +1543,10 @@ ifpga_rawdev_create(struct rte_pci_device *pci_dev, if (ret) goto free_adapter_data; + ret = ifpga_monitor_start_func(dev); + if (ret) + goto free_adapter_data; + return ret; free_adapter_data: @@ -1597,7 +1622,7 @@ ifpga_rawdev_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, static int ifpga_rawdev_pci_remove(struct rte_pci_device *pci_dev) { - ifpga_monitor_stop_func(); + IFPGA_RAWDEV_PMD_INFO("remove pci_dev %s", pci_dev->device.name); return ifpga_rawdev_destroy(pci_dev); } @@ -1648,7 +1673,6 @@ ifpga_cfg_probe(struct rte_vdev_device *dev) struct ifpga_rawdev *ifpga_dev; int port; char *name = NULL; - const char *bdf; char dev_name[RTE_RAWDEV_NAME_MAX_LEN]; int ret = -1; @@ -1697,10 +1721,6 @@ ifpga_cfg_probe(struct rte_vdev_device *dev) ifpga_dev = ifpga_rawdev_get(rawdev); if (!ifpga_dev) goto end; - bdf = name; - ifpga_rawdev_fill_info(ifpga_dev, bdf); - - ifpga_monitor_start_func(); memset(dev_name, 0, sizeof(dev_name)); snprintf(dev_name, RTE_RAWDEV_NAME_MAX_LEN, "%d|%s", diff --git a/drivers/raw/ifpga/ifpga_rawdev.h b/drivers/raw/ifpga/ifpga_rawdev.h index 5c3ecd35d6..f082280f9a 100644 --- a/drivers/raw/ifpga/ifpga_rawdev.h +++ b/drivers/raw/ifpga/ifpga_rawdev.h @@ -59,6 +59,8 @@ struct ifpga_rawdev { uint32_t aer_old[2]; char fvl_bdf[8][16]; char parent_bdf[16]; + /* enable monitor thread poll device's sensors or not */ + int poll_enabled; }; struct ifpga_rawdev * -- 2.30.2 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2022-03-09 16:30:09.311744195 +0000 +++ 0021-raw-ifpga-fix-monitor-thread.patch 2022-03-09 16:30:08.551025243 +0000 @@ -1 +1 @@ -From 2479a1e9a8886c93ba4c8a7aa2a35a04302f1aae Mon Sep 17 00:00:00 2001 +From 4297b2d419135300f90ce52328d384de484c9d67 Mon Sep 17 00:00:00 2001 @@ -5,0 +6,2 @@ +[ upstream commit 2479a1e9a8886c93ba4c8a7aa2a35a04302f1aae ] + @@ -14 +15,0 @@ -Cc: stable@dpdk.org @@ -24 +25 @@ -index cc569c5e71..26c1366a64 100644 +index decf183a0b..fb37313784 100644 @@ -35,2 +36,4 @@ - static struct ifpga_rawdev * -@@ -133,6 +133,7 @@ ifpga_rawdev_allocate(struct rte_rawdev *rawdev) + #define IFPGA_MAX_IRQ 12 +@@ -134,6 +134,7 @@ ifpga_rawdev_allocate(struct rte_rawdev *rawdev) + dev = &ifpga_rawdevices[dev_id]; + dev->rawdev = rawdev; @@ -38,2 +40,0 @@ - for (i = 0; i < IFPGA_MAX_IRQ; i++) - dev->intr_handle[i] = NULL; @@ -44 +45 @@ -@@ -207,10 +208,11 @@ static int ifpga_get_dev_vendor_id(const char *bdf, +@@ -208,10 +209,11 @@ static int ifpga_get_dev_vendor_id(const char *bdf, @@ -59 +60 @@ -@@ -225,7 +227,11 @@ static int ifpga_rawdev_fill_info(struct ifpga_rawdev *ifpga_dev, +@@ -226,7 +228,11 @@ static int ifpga_rawdev_fill_info(struct ifpga_rawdev *ifpga_dev, @@ -72 +73 @@ -@@ -375,7 +381,7 @@ ifpga_monitor_sensor(struct rte_rawdev *raw_dev, +@@ -376,7 +382,7 @@ ifpga_monitor_sensor(struct rte_rawdev *raw_dev, @@ -81 +82 @@ -@@ -417,7 +423,7 @@ static int set_surprise_link_check_aer( +@@ -418,7 +424,7 @@ static int set_surprise_link_check_aer( @@ -90 +91 @@ -@@ -496,11 +502,11 @@ ifpga_rawdev_gsd_handle(__rte_unused void *param) +@@ -497,11 +503,11 @@ ifpga_rawdev_gsd_handle(__rte_unused void *param) @@ -104 +105 @@ -@@ -520,32 +526,46 @@ ifpga_rawdev_gsd_handle(__rte_unused void *param) +@@ -521,32 +527,46 @@ ifpga_rawdev_gsd_handle(__rte_unused void *param) @@ -124,4 +125,4 @@ - ret = rte_ctrl_thread_create(&ifpga_monitor_start_thread, - "ifpga-monitor", NULL, - ifpga_rawdev_gsd_handle, NULL); - if (ret != 0) { + ret = pthread_create(&ifpga_monitor_start_thread, + NULL, + ifpga_rawdev_gsd_handle, NULL); + if (ret) { @@ -157 +158 @@ -@@ -718,6 +738,7 @@ ifpga_rawdev_close(struct rte_rawdev *dev) +@@ -719,6 +739,7 @@ ifpga_rawdev_close(struct rte_rawdev *dev) @@ -165 +166 @@ -@@ -1571,6 +1592,10 @@ ifpga_rawdev_create(struct rte_pci_device *pci_dev, +@@ -1522,6 +1543,10 @@ ifpga_rawdev_create(struct rte_pci_device *pci_dev, @@ -176 +177 @@ -@@ -1646,7 +1671,7 @@ ifpga_rawdev_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, +@@ -1597,7 +1622,7 @@ ifpga_rawdev_pci_probe(struct rte_pci_driver *pci_drv __rte_unused, @@ -185 +186 @@ -@@ -1697,7 +1722,6 @@ ifpga_cfg_probe(struct rte_vdev_device *dev) +@@ -1648,7 +1673,6 @@ ifpga_cfg_probe(struct rte_vdev_device *dev) @@ -193 +194 @@ -@@ -1746,10 +1770,6 @@ ifpga_cfg_probe(struct rte_vdev_device *dev) +@@ -1697,10 +1721,6 @@ ifpga_cfg_probe(struct rte_vdev_device *dev) @@ -205 +206 @@ -index 6e09afead3..857b73463d 100644 +index 5c3ecd35d6..f082280f9a 100644 @@ -208 +209,3 @@ -@@ -62,6 +62,8 @@ struct ifpga_rawdev { +@@ -59,6 +59,8 @@ struct ifpga_rawdev { + uint32_t aer_old[2]; + char fvl_bdf[8][16]; @@ -210,2 +212,0 @@ - /* 0 for FME interrupt, others are reserved for AFU irq */ - void *intr_handle[IFPGA_MAX_IRQ];