From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B38E1A0542 for ; Fri, 18 Nov 2022 13:32:45 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 88A8D4021F; Fri, 18 Nov 2022 13:32:45 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 256674003F for ; Fri, 18 Nov 2022 13:32:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1668774762; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+X6rkoLxsUOR6KV6g21Tgik7cGDcPsqpXanDO5k2Xj0=; b=NbAznnptzT9kTPPxlYE1PiCHnihMFyN75bVhwVhXyKKljbAWzGvMPEGlcJh9X6WRs7Mthr +D6FZr5QVyZOkWJu9L3IK5757eKxJk3lA/IG3h5VgXEcY8WPqR09vK8jNdICQ2XOwwYZfK IO8uv2zvDvevWQsKkRW7/MrKLvAYnDA= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-240-gmsHu1ylNaeXNagotDq4wQ-1; Fri, 18 Nov 2022 07:32:41 -0500 X-MC-Unique: gmsHu1ylNaeXNagotDq4wQ-1 Received: by mail-wr1-f70.google.com with SMTP id v14-20020adf8b4e000000b0024174021277so1527974wra.13 for ; Fri, 18 Nov 2022 04:32:41 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:subject:references:cc:to:from :content-language:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+X6rkoLxsUOR6KV6g21Tgik7cGDcPsqpXanDO5k2Xj0=; b=eD++X8RjDyt11V7pTItVnPyJ4hOaDucMZArnoJeQZKdj+RvmY29fTCpc7h/CZV4GB/ 9zdIPz8g9CygyOefvHEhFGJB6oi+qQ4NuH0adYCWx4kZt6VINODX+jxMP09Pzqulho2h NrWHClCOrZRCn1YZ7VYcrNvqqa6lrZ7wk+2H5YQ3u1utNyHswhCTKjBdBkioGzWlNOid KrYcA3YheTsWGCw06OlZcQg8KKG/aLPFz4A48/5w+PbnfgLvNIPMncQTjNKqe/HCRhcA k4zkpgzdV9vMpc3j+hoLAWR5FZaQLMmbLztr0OPoEf5SmxjRom7aTMAFdC8p4fEi1gBt BIFw== X-Gm-Message-State: ANoB5pmg/2QPqW5ByUORbYJHem0cUi3c7iK4hD51yzD+h0308uCiT9yV 3EUpzsGMSbjcyPQ2U/bseL/T4akxLL4PjoXxyQb6PBDnT2mjfW0n+oJkyFcJjha4P/dbj6DST5L IRgqWKnQ= X-Received: by 2002:a05:600c:3501:b0:3c7:4f0:77b3 with SMTP id h1-20020a05600c350100b003c704f077b3mr8189211wmq.93.1668774760249; Fri, 18 Nov 2022 04:32:40 -0800 (PST) X-Google-Smtp-Source: AA0mqf679M+87xzewFi44XVUGMSdGoIKucpdC3YpQHc/qVhWdJc6phIZnkb2E64A8WBdZOaBiKyPWw== X-Received: by 2002:a05:600c:3501:b0:3c7:4f0:77b3 with SMTP id h1-20020a05600c350100b003c704f077b3mr8189201wmq.93.1668774760045; Fri, 18 Nov 2022 04:32:40 -0800 (PST) Received: from [192.168.0.36] ([78.19.102.117]) by smtp.gmail.com with ESMTPSA id g17-20020a05600c311100b003a2f2bb72d5sm11202030wmo.45.2022.11.18.04.32.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 18 Nov 2022 04:32:39 -0800 (PST) Message-ID: <7e29a23f-9bb2-9ee8-e0ee-2814b48d013a@redhat.com> Date: Fri, 18 Nov 2022 12:32:38 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.1 From: Kevin Traynor To: Yiding Zhou Cc: Qi Zhang , dpdk stable References: <20221111103337.307408-1-ktraynor@redhat.com> <20221111103337.307408-28-ktraynor@redhat.com> Subject: Re: patch 'net/iavf: add thread for event callbacks' has been queued to stable release 21.11.3 In-Reply-To: <20221111103337.307408-28-ktraynor@redhat.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org On 11/11/2022 10:33, Kevin Traynor wrote: > Hi, > > FYI, your patch has been queued to stable release 21.11.3 > > Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. > It will be pushed if I get no objections before 11/14/22. So please > shout if anyone has objections. > This patch is causing a build failure on Windows [1]. I won't push it to to the dpdk-stable 21.11 branch. Please send a rebased version, or let me know if it should be dropped from 21.11. > Also note that after the patch there's a diff of the upstream commit vs the > patch applied to the branch. This will indicate if there was any rebasing > needed to apply to the stable branch. If there were code changes for rebasing > (ie: not only metadata diffs), please double check that the rebase was > correctly done. > > Queued patches are on a temporary branch at: > https://github.com/kevintraynor/dpdk-stable > > This queued commit can be viewed at: > https://github.com/kevintraynor/dpdk-stable/commit/c346009ca9686fabd300f8afe688239426b5eebd > > Thanks. > > Kevin > > --- > From c346009ca9686fabd300f8afe688239426b5eebd Mon Sep 17 00:00:00 2001 > From: Yiding Zhou > Date: Thu, 20 Oct 2022 13:00:22 +0800 > Subject: [PATCH] net/iavf: add thread for event callbacks > > [ upstream commit cb5c1b91f76f436724cd09f26c7432b2775b519c ] > > All callbacks registered for ethdev events are called in > eal-intr-thread, and some of them execute virtchnl commands. > Because interrupts are disabled in the intr thread, no response > will be received for these commands. So all callbacks should > be called in a new context. > > When the device is bonded, the bond pmd registers a callback for > the LSC event to execute virtchnl commands to reinitialize the > device, and it would also raise the above issue. > > This commit adds a new thread to call all event callbacks. > > Fixes: 48de41ca11f0 ("net/avf: enable link status update") > Fixes: 84108425054a ("net/iavf: support asynchronous virtual channel message") > > Signed-off-by: Yiding Zhou > Acked-by: Qi Zhang > --- > drivers/net/iavf/iavf.h | 2 + > drivers/net/iavf/iavf_ethdev.c | 5 ++ > drivers/net/iavf/iavf_vchnl.c | 153 +++++++++++++++++++++++++++++++-- > 3 files changed, 154 insertions(+), 6 deletions(-) > > diff --git a/drivers/net/iavf/iavf.h b/drivers/net/iavf/iavf.h > index 29692e3994..afffc1a13e 100644 > --- a/drivers/net/iavf/iavf.h > +++ b/drivers/net/iavf/iavf.h > @@ -401,4 +401,6 @@ _atomic_set_async_response_cmd(struct iavf_info *vf, enum virtchnl_ops ops) > int iavf_check_api_version(struct iavf_adapter *adapter); > int iavf_get_vf_resource(struct iavf_adapter *adapter); > +void iavf_dev_event_handler_fini(void); > +int iavf_dev_event_handler_init(void); > void iavf_handle_virtchnl_msg(struct rte_eth_dev *dev); > int iavf_enable_vlan_strip(struct iavf_adapter *adapter); > diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c > index f835457e4f..11ddd6dc16 100644 > --- a/drivers/net/iavf/iavf_ethdev.c > +++ b/drivers/net/iavf/iavf_ethdev.c > @@ -2562,4 +2562,7 @@ iavf_dev_init(struct rte_eth_dev *eth_dev) > ð_dev->data->mac_addrs[0]); > > + if (iavf_dev_event_handler_init()) > + goto init_vf_err; > + > if (vf->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_WB_ON_ITR) { > /* register callback func to eal lib */ > @@ -2716,4 +2719,6 @@ iavf_dev_uninit(struct rte_eth_dev *dev) > iavf_dev_close(dev); > > + iavf_dev_event_handler_fini(); > + > return 0; > } > diff --git a/drivers/net/iavf/iavf_vchnl.c b/drivers/net/iavf/iavf_vchnl.c > index 1bd3559ec2..54c6e39e16 100644 > --- a/drivers/net/iavf/iavf_vchnl.c > +++ b/drivers/net/iavf/iavf_vchnl.c > @@ -3,4 +3,5 @@ > */ > > +#include > #include > #include > @@ -12,4 +13,5 @@ > #include > #include > +#include > > #include > @@ -28,4 +30,144 @@ > #define ASQ_DELAY_MS 1 > > +#define MAX_EVENT_PENDING 16 > + > +struct iavf_event_element { > + TAILQ_ENTRY(iavf_event_element) next; > + struct rte_eth_dev *dev; > + enum rte_eth_event_type event; > + void *param; > + size_t param_alloc_size; > + uint8_t param_alloc_data[0]; > +}; > + > +struct iavf_event_handler { > + uint32_t ndev; > + pthread_t tid; > + int fd[2]; > + pthread_mutex_t lock; > + TAILQ_HEAD(event_list, iavf_event_element) pending; > +}; > + > +static struct iavf_event_handler event_handler = { > + .fd = {-1, -1}, > +}; > + > +#ifndef TAILQ_FOREACH_SAFE > +#define TAILQ_FOREACH_SAFE(var, head, field, tvar) \ > + for ((var) = TAILQ_FIRST((head)); \ > + (var) && ((tvar) = TAILQ_NEXT((var), field), 1); \ > + (var) = (tvar)) > +#endif > + > +static void * > +iavf_dev_event_handle(void *param __rte_unused) > +{ > + struct iavf_event_handler *handler = &event_handler; > + TAILQ_HEAD(event_list, iavf_event_element) pending; > + > + while (true) { > + char unused[MAX_EVENT_PENDING]; > + ssize_t nr = read(handler->fd[0], &unused, sizeof(unused)); > + if (nr <= 0) > + break; > + > + TAILQ_INIT(&pending); > + pthread_mutex_lock(&handler->lock); > + TAILQ_CONCAT(&pending, &handler->pending, next); > + pthread_mutex_unlock(&handler->lock); > + > + struct iavf_event_element *pos, *save_next; > + TAILQ_FOREACH_SAFE(pos, &pending, next, save_next) { > + TAILQ_REMOVE(&pending, pos, next); > + rte_eth_dev_callback_process(pos->dev, pos->event, pos->param); > + rte_free(pos); > + } > + } > + > + return NULL; > +} > + > +static void > +iavf_dev_event_post(struct rte_eth_dev *dev, > + enum rte_eth_event_type event, > + void *param, size_t param_alloc_size) > +{ > + struct iavf_event_handler *handler = &event_handler; > + char notify_byte; > + struct iavf_event_element *elem = rte_malloc(NULL, sizeof(*elem) + param_alloc_size, 0); > + if (!elem) > + return; > + > + elem->dev = dev; > + elem->event = event; > + elem->param = param; > + elem->param_alloc_size = param_alloc_size; > + if (param && param_alloc_size) { > + rte_memcpy(elem->param_alloc_data, param, param_alloc_size); > + elem->param = elem->param_alloc_data; > + } > + > + pthread_mutex_lock(&handler->lock); > + TAILQ_INSERT_TAIL(&handler->pending, elem, next); > + pthread_mutex_unlock(&handler->lock); > + > + ssize_t nw = write(handler->fd[1], ¬ify_byte, 1); > + RTE_SET_USED(nw); > +} > + > +int > +iavf_dev_event_handler_init(void) > +{ > + struct iavf_event_handler *handler = &event_handler; > + > + if (__atomic_add_fetch(&handler->ndev, 1, __ATOMIC_RELAXED) != 1) > + return 0; > +#if defined(RTE_EXEC_ENV_IS_WINDOWS) && RTE_EXEC_ENV_IS_WINDOWS != 0 > + int err = _pipe(handler->fd, MAX_EVENT_PENDING, O_BINARY); > +#else > + int err = pipe(handler->fd); > +#endif ../drivers/net/iavf/iavf_vchnl.c:128:12: error: implicit declaration of function 'pipe' is invalid in C99 [-Werror,-Wimplicit-function-declaration] int err = pipe(handler->fd); ^ ../drivers/net/iavf/iavf_vchnl.c:128:12: note: did you mean '_pipe'? C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\ucrt\corecrt_io.h:241:30: note: '_pipe' declared here _DCRTIMP int __cdecl _pipe( From: https://dpdkdashboard.iol.unh.edu/results/dashboard/tarballs/22355/ > + if (err != 0) { > + __atomic_sub_fetch(&handler->ndev, 1, __ATOMIC_RELAXED); > + return -1; > + } > + > + TAILQ_INIT(&handler->pending); > + pthread_mutex_init(&handler->lock, NULL); > + > + if (rte_ctrl_thread_create(&handler->tid, "iavf-event-thread", > + NULL, iavf_dev_event_handle, NULL)) { > + __atomic_sub_fetch(&handler->ndev, 1, __ATOMIC_RELAXED); > + return -1; > + } > + > + return 0; > +} > + > +void > +iavf_dev_event_handler_fini(void) > +{ > + struct iavf_event_handler *handler = &event_handler; > + > + if (__atomic_sub_fetch(&handler->ndev, 1, __ATOMIC_RELAXED) != 0) > + return; > + > + int unused = pthread_cancel(handler->tid); > + RTE_SET_USED(unused); > + close(handler->fd[0]); > + close(handler->fd[1]); > + handler->fd[0] = -1; > + handler->fd[1] = -1; > + > + pthread_join(handler->tid, NULL); > + pthread_mutex_destroy(&handler->lock); > + > + struct iavf_event_element *pos, *save_next; > + TAILQ_FOREACH_SAFE(pos, &handler->pending, next, save_next) { > + TAILQ_REMOVE(&handler->pending, pos, next); > + rte_free(pos); > + } > +} > + > static uint32_t > iavf_convert_link_speed(enum virtchnl_link_speed virt_link_speed) > @@ -279,6 +421,6 @@ iavf_handle_pf_event_msg(struct rte_eth_dev *dev, uint8_t *msg, > PMD_DRV_LOG(DEBUG, "VIRTCHNL_EVENT_RESET_IMPENDING event"); > vf->vf_reset = true; > - rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_RESET, > - NULL); > + iavf_dev_event_post(dev, RTE_ETH_EVENT_INTR_RESET, > + NULL, 0); > break; > case VIRTCHNL_EVENT_LINK_CHANGE: > @@ -294,5 +436,5 @@ iavf_handle_pf_event_msg(struct rte_eth_dev *dev, uint8_t *msg, > } > iavf_dev_link_update(dev, 0); > - rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_LSC, NULL); > + iavf_dev_event_post(dev, RTE_ETH_EVENT_INTR_LSC, NULL, 0); > break; > case VIRTCHNL_EVENT_PF_DRIVER_CLOSE: > @@ -360,7 +502,6 @@ iavf_handle_virtchnl_msg(struct rte_eth_dev *dev) > RTE_ETH_EVENT_IPSEC_UNKNOWN; > desc.metadata = ev->ipsec_event_data; > - rte_eth_dev_callback_process(dev, > - RTE_ETH_EVENT_IPSEC, > - &desc); > + iavf_dev_event_post(dev, RTE_ETH_EVENT_IPSEC, > + &desc, sizeof(desc)); > return; > }