From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1B80CA0543 for ; Sat, 5 Nov 2022 18:13:36 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 16D9B42D29; Sat, 5 Nov 2022 18:13:36 +0100 (CET) Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) by mails.dpdk.org (Postfix) with ESMTP id 52595400D5 for ; Sat, 5 Nov 2022 18:13:35 +0100 (CET) Received: by mail-wr1-f43.google.com with SMTP id bk15so10833331wrb.13 for ; Sat, 05 Nov 2022 10:13:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6H9HHfn2I4fyW/lb85zev2jbNpU79yy1vOQESUNVNlY=; b=brfz80Y7Y/KQRN2sCw/E8g0IdzTjK/O6kbhG5EOwi8gmS3AgmK12+DlsgZyWH1muAQ NR21D94B9SQ0WAaYsjtGTyCV+mRUdc9N859tv6Cm0l78R0s0E/LsQ1uzmrtY6rLTSdTZ 6CfG2y1GOGMCfkxYSSR6e5SvYYMlBSt6rfjmsqdjBOQ/6oWkMnQ2+klw7yFS6P7AGSbk 5x2O+Yt+/iN276tlzuBZIbWz962KcH3SV4WITFbSu/22h8Fstc/QbD6kdXAd6BwUnrJZ UBlU5Lk/BYsq3M4ewLoHjAav0uVNuMo8P41pDA5YkBDENQrQ5fctJ2ZUDCTEWLjYUy1F akXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6H9HHfn2I4fyW/lb85zev2jbNpU79yy1vOQESUNVNlY=; b=Psduugyl5+CAgsR12b5S7I5SGjvPzaoRca6O7kWO7RU5V/jruP/cSTlU1KBkHpMj8O wKttYankqarjIZ3bGtky55txepcRXn8JICuICazXpk1ZqMbcaG2F6d5n4q/Xsq8gqWBD FmVwfNoAqLykCVyo2dFtw4nlGccdzGEzYUzqnjh0+DYMy0Z4l4pMnzwcZBx5vPVryRYf QQkOhkCE87VAj/OhSz9JnaKILQDEfr7s6nJuv2aSgoDGIJRR4l1Y1vC1AdL4E4BTP70P ZhBkZIbh0g2K2Vv0gJKMSmJ05MqkhdyoEzfqD0+TxkS2Hr3K5wV4mef8epY9/4RHGsiu t6kQ== X-Gm-Message-State: ACrzQf3rwGkSDDf0YnfXe1tGG8vejFNYf2hOTWCw4HM9kqTNsFBtPecv 4S3zqqEDSrgGF2MqvvL0n0d0zAB5NeY= X-Google-Smtp-Source: AMsMyM5FsUaCducNIChMSJw7NFVD5oxmLvqtw2vOB/LH/Zd32WnQITkGYmQS1Ct0CZfFgsxQ/VYknw== X-Received: by 2002:a5d:68ce:0:b0:236:9981:d216 with SMTP id p14-20020a5d68ce000000b002369981d216mr25868548wrw.384.1667668414961; Sat, 05 Nov 2022 10:13:34 -0700 (PDT) Received: from localhost ([2a01:4b00:d307:1000:f1d3:eb5e:11f4:a7d9]) by smtp.gmail.com with ESMTPSA id m11-20020a5d4a0b000000b0022ca921dc67sm2539784wrq.88.2022.11.05.10.13.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 05 Nov 2022 10:13:34 -0700 (PDT) From: luca.boccassi@gmail.com To: Yiding Zhou Cc: Qi Zhang , dpdk stable Subject: patch 'net/iavf: add thread for event callbacks' has been queued to stable release 20.11.7 Date: Sat, 5 Nov 2022 17:11:31 +0000 Message-Id: <20221105171146.1520039-32-luca.boccassi@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221105171146.1520039-1-luca.boccassi@gmail.com> References: <20221103092758.1099402-100-luca.boccassi@gmail.com> <20221105171146.1520039-1-luca.boccassi@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Hi, FYI, your patch has been queued to stable release 20.11.7 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 11/07/22. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://github.com/kevintraynor/dpdk-stable This queued commit can be viewed at: https://github.com/kevintraynor/dpdk-stable/commit/6932f663ad5c5d163678aaf98c92cac4af077111 Thanks. Luca Boccassi --- >From 6932f663ad5c5d163678aaf98c92cac4af077111 Mon Sep 17 00:00:00 2001 From: Yiding Zhou Date: Thu, 20 Oct 2022 13:00:22 +0800 Subject: [PATCH] net/iavf: add thread for event callbacks [ upstream commit cb5c1b91f76f436724cd09f26c7432b2775b519c ] All callbacks registered for ethdev events are called in eal-intr-thread, and some of them execute virtchnl commands. Because interrupts are disabled in the intr thread, no response will be received for these commands. So all callbacks should be called in a new context. When the device is bonded, the bond pmd registers a callback for the LSC event to execute virtchnl commands to reinitialize the device, and it would also raise the above issue. This commit adds a new thread to call all event callbacks. Fixes: 48de41ca11f0 ("net/avf: enable link status update") Fixes: 84108425054a ("net/iavf: support asynchronous virtual channel message") Signed-off-by: Yiding Zhou Acked-by: Qi Zhang --- drivers/net/iavf/iavf.h | 2 + drivers/net/iavf/iavf_ethdev.c | 5 ++ drivers/net/iavf/iavf_vchnl.c | 147 ++++++++++++++++++++++++++++++++- 3 files changed, 151 insertions(+), 3 deletions(-) diff --git a/drivers/net/iavf/iavf.h b/drivers/net/iavf/iavf.h index bc49f1fa17..41d9856358 100644 --- a/drivers/net/iavf/iavf.h +++ b/drivers/net/iavf/iavf.h @@ -287,6 +287,8 @@ _atomic_set_cmd(struct iavf_info *vf, enum virtchnl_ops ops) int iavf_check_api_version(struct iavf_adapter *adapter); int iavf_get_vf_resource(struct iavf_adapter *adapter); +void iavf_dev_event_handler_fini(void); +int iavf_dev_event_handler_init(void); void iavf_handle_virtchnl_msg(struct rte_eth_dev *dev); int iavf_enable_vlan_strip(struct iavf_adapter *adapter); int iavf_disable_vlan_strip(struct iavf_adapter *adapter); diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c index 060b3e3707..9387f17c81 100644 --- a/drivers/net/iavf/iavf_ethdev.c +++ b/drivers/net/iavf/iavf_ethdev.c @@ -2034,6 +2034,9 @@ iavf_dev_init(struct rte_eth_dev *eth_dev) rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr, ð_dev->data->mac_addrs[0]); + if (iavf_dev_event_handler_init()) + return 0; + /* register callback func to eal lib */ rte_intr_callback_register(&pci_dev->intr_handle, iavf_dev_interrupt_handler, @@ -2121,6 +2124,8 @@ iavf_dev_uninit(struct rte_eth_dev *dev) iavf_dev_close(dev); + iavf_dev_event_handler_fini(); + return 0; } diff --git a/drivers/net/iavf/iavf_vchnl.c b/drivers/net/iavf/iavf_vchnl.c index f6da2cf4bd..77f10a90c7 100644 --- a/drivers/net/iavf/iavf_vchnl.c +++ b/drivers/net/iavf/iavf_vchnl.c @@ -2,6 +2,7 @@ * Copyright(c) 2017 Intel Corporation */ +#include #include #include #include @@ -26,6 +27,146 @@ #define MAX_TRY_TIMES 200 #define ASQ_DELAY_MS 10 +#define MAX_EVENT_PENDING 16 + +struct iavf_event_element { + TAILQ_ENTRY(iavf_event_element) next; + struct rte_eth_dev *dev; + enum rte_eth_event_type event; + void *param; + size_t param_alloc_size; + uint8_t param_alloc_data[0]; +}; + +struct iavf_event_handler { + uint32_t ndev; + pthread_t tid; + int fd[2]; + pthread_mutex_t lock; + TAILQ_HEAD(event_list, iavf_event_element) pending; +}; + +static struct iavf_event_handler event_handler = { + .fd = {-1, -1}, +}; + +#ifndef TAILQ_FOREACH_SAFE +#define TAILQ_FOREACH_SAFE(var, head, field, tvar) \ + for ((var) = TAILQ_FIRST((head)); \ + (var) && ((tvar) = TAILQ_NEXT((var), field), 1); \ + (var) = (tvar)) +#endif + +static void * +iavf_dev_event_handle(void *param __rte_unused) +{ + struct iavf_event_handler *handler = &event_handler; + TAILQ_HEAD(event_list, iavf_event_element) pending; + + while (true) { + char unused[MAX_EVENT_PENDING]; + ssize_t nr = read(handler->fd[0], &unused, sizeof(unused)); + if (nr <= 0) + break; + + TAILQ_INIT(&pending); + pthread_mutex_lock(&handler->lock); + TAILQ_CONCAT(&pending, &handler->pending, next); + pthread_mutex_unlock(&handler->lock); + + struct iavf_event_element *pos, *save_next; + TAILQ_FOREACH_SAFE(pos, &pending, next, save_next) { + TAILQ_REMOVE(&pending, pos, next); + rte_eth_dev_callback_process(pos->dev, pos->event, pos->param); + rte_free(pos); + } + } + + return NULL; +} + +static void +iavf_dev_event_post(struct rte_eth_dev *dev, + enum rte_eth_event_type event, + void *param, size_t param_alloc_size) +{ + struct iavf_event_handler *handler = &event_handler; + char notify_byte; + struct iavf_event_element *elem = rte_malloc(NULL, sizeof(*elem) + param_alloc_size, 0); + if (!elem) + return; + + elem->dev = dev; + elem->event = event; + elem->param = param; + elem->param_alloc_size = param_alloc_size; + if (param && param_alloc_size) { + rte_memcpy(elem->param_alloc_data, param, param_alloc_size); + elem->param = elem->param_alloc_data; + } + + pthread_mutex_lock(&handler->lock); + TAILQ_INSERT_TAIL(&handler->pending, elem, next); + pthread_mutex_unlock(&handler->lock); + + ssize_t nw = write(handler->fd[1], ¬ify_byte, 1); + RTE_SET_USED(nw); +} + +int +iavf_dev_event_handler_init(void) +{ + struct iavf_event_handler *handler = &event_handler; + + if (__atomic_add_fetch(&handler->ndev, 1, __ATOMIC_RELAXED) != 1) + return 0; +#if defined(RTE_EXEC_ENV_IS_WINDOWS) && RTE_EXEC_ENV_IS_WINDOWS != 0 + int err = _pipe(handler->fd, MAX_EVENT_PENDING, O_BINARY); +#else + int err = pipe(handler->fd); +#endif + if (err != 0) { + __atomic_sub_fetch(&handler->ndev, 1, __ATOMIC_RELAXED); + return -1; + } + + TAILQ_INIT(&handler->pending); + pthread_mutex_init(&handler->lock, NULL); + + if (rte_ctrl_thread_create(&handler->tid, "iavf-event-thread", + NULL, iavf_dev_event_handle, NULL)) { + __atomic_sub_fetch(&handler->ndev, 1, __ATOMIC_RELAXED); + return -1; + } + + return 0; +} + +void +iavf_dev_event_handler_fini(void) +{ + struct iavf_event_handler *handler = &event_handler; + + if (__atomic_sub_fetch(&handler->ndev, 1, __ATOMIC_RELAXED) != 0) + return; + + int unused = pthread_cancel(handler->tid); + RTE_SET_USED(unused); + close(handler->fd[0]); + close(handler->fd[1]); + handler->fd[0] = -1; + handler->fd[1] = -1; + + pthread_join(handler->tid, NULL); + pthread_mutex_destroy(&handler->lock); + + struct iavf_event_element *pos, *save_next; + TAILQ_FOREACH_SAFE(pos, &handler->pending, next, save_next) { + TAILQ_REMOVE(&handler->pending, pos, next); + rte_free(pos); + } +} + static uint32_t iavf_convert_link_speed(enum virtchnl_link_speed virt_link_speed) { @@ -262,8 +403,8 @@ iavf_handle_pf_event_msg(struct rte_eth_dev *dev, uint8_t *msg, case VIRTCHNL_EVENT_RESET_IMPENDING: PMD_DRV_LOG(DEBUG, "VIRTCHNL_EVENT_RESET_IMPENDING event"); vf->vf_reset = true; - rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_RESET, - NULL); + iavf_dev_event_post(dev, RTE_ETH_EVENT_INTR_RESET, + NULL, 0); break; case VIRTCHNL_EVENT_LINK_CHANGE: PMD_DRV_LOG(DEBUG, "VIRTCHNL_EVENT_LINK_CHANGE event"); @@ -277,7 +418,7 @@ iavf_handle_pf_event_msg(struct rte_eth_dev *dev, uint8_t *msg, vf->link_speed = iavf_convert_link_speed(speed); } iavf_dev_link_update(dev, 0); - rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_LSC, NULL); + iavf_dev_event_post(dev, RTE_ETH_EVENT_INTR_LSC, NULL, 0); break; case VIRTCHNL_EVENT_PF_DRIVER_CLOSE: PMD_DRV_LOG(DEBUG, "VIRTCHNL_EVENT_PF_DRIVER_CLOSE event"); -- 2.34.1 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2022-11-05 17:11:11.499928021 +0000 +++ 0032-net-iavf-add-thread-for-event-callbacks.patch 2022-11-05 17:11:08.750944273 +0000 @@ -1 +1 @@ -From cb5c1b91f76f436724cd09f26c7432b2775b519c Mon Sep 17 00:00:00 2001 +From 6932f663ad5c5d163678aaf98c92cac4af077111 Mon Sep 17 00:00:00 2001 @@ -5,0 +6,2 @@ +[ upstream commit cb5c1b91f76f436724cd09f26c7432b2775b519c ] + @@ -20 +21,0 @@ -Cc: stable@dpdk.org @@ -27,2 +28,2 @@ - drivers/net/iavf/iavf_vchnl.c | 153 +++++++++++++++++++++++++++++++-- - 3 files changed, 154 insertions(+), 6 deletions(-) + drivers/net/iavf/iavf_vchnl.c | 147 ++++++++++++++++++++++++++++++++- + 3 files changed, 151 insertions(+), 3 deletions(-) @@ -31 +32 @@ -index 26b858f6f0..1edebab8dc 100644 +index bc49f1fa17..41d9856358 100644 @@ -34,2 +35,2 @@ -@@ -424,6 +424,8 @@ _atomic_set_async_response_cmd(struct iavf_info *vf, enum virtchnl_ops ops) - } +@@ -287,6 +287,8 @@ _atomic_set_cmd(struct iavf_info *vf, enum virtchnl_ops ops) + @@ -44 +45 @@ -index b1958e0474..8d9f3a6d3d 100644 +index 060b3e3707..9387f17c81 100644 @@ -47 +48 @@ -@@ -2633,6 +2633,9 @@ iavf_dev_init(struct rte_eth_dev *eth_dev) +@@ -2034,6 +2034,9 @@ iavf_dev_init(struct rte_eth_dev *eth_dev) @@ -52 +53 @@ -+ goto init_vf_err; ++ return 0; @@ -54,4 +55,4 @@ - if (vf->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_WB_ON_ITR) { - /* register callback func to eal lib */ - rte_intr_callback_register(pci_dev->intr_handle, -@@ -2787,6 +2790,8 @@ iavf_dev_uninit(struct rte_eth_dev *dev) + /* register callback func to eal lib */ + rte_intr_callback_register(&pci_dev->intr_handle, + iavf_dev_interrupt_handler, +@@ -2121,6 +2124,8 @@ iavf_dev_uninit(struct rte_eth_dev *dev) @@ -67 +68 @@ -index 729400cb91..654bc7edb6 100644 +index f6da2cf4bd..77f10a90c7 100644 @@ -78,11 +79,3 @@ -@@ -11,6 +12,7 @@ - #include - #include - #include -+#include - - #include - #include -@@ -27,6 +29,146 @@ - #define MAX_TRY_TIMES 2000 - #define ASQ_DELAY_MS 1 +@@ -26,6 +27,146 @@ + #define MAX_TRY_TIMES 200 + #define ASQ_DELAY_MS 10 @@ -233 +226 @@ -@@ -278,8 +420,8 @@ iavf_handle_pf_event_msg(struct rte_eth_dev *dev, uint8_t *msg, +@@ -262,8 +403,8 @@ iavf_handle_pf_event_msg(struct rte_eth_dev *dev, uint8_t *msg, @@ -244 +237 @@ -@@ -293,7 +435,7 @@ iavf_handle_pf_event_msg(struct rte_eth_dev *dev, uint8_t *msg, +@@ -277,7 +418,7 @@ iavf_handle_pf_event_msg(struct rte_eth_dev *dev, uint8_t *msg, @@ -253,12 +245,0 @@ -@@ -359,9 +501,8 @@ iavf_handle_virtchnl_msg(struct rte_eth_dev *dev) - desc.subtype = - RTE_ETH_EVENT_IPSEC_UNKNOWN; - desc.metadata = ev->ipsec_event_data; -- rte_eth_dev_callback_process(dev, -- RTE_ETH_EVENT_IPSEC, -- &desc); -+ iavf_dev_event_post(dev, RTE_ETH_EVENT_IPSEC, -+ &desc, sizeof(desc)); - return; - } -