From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5F99942588; Wed, 13 Sep 2023 13:45:50 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 72642402CF; Wed, 13 Sep 2023 13:45:49 +0200 (CEST) Received: from wout5-smtp.messagingengine.com (wout5-smtp.messagingengine.com [64.147.123.21]) by mails.dpdk.org (Postfix) with ESMTP id 6225C40293 for ; Wed, 13 Sep 2023 13:45:47 +0200 (CEST) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id B6F2E3200979; Wed, 13 Sep 2023 07:45:43 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Wed, 13 Sep 2023 07:45:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1694605543; x=1694691943; bh=lBicXkbl7p6iD8AYhNBdzSBYrbBSMa+/z2j JChdqbe0=; b=erKXfBbz+wibtkFS0QFZeh/wNUfSRZtHa1mP9avd7wj7RuMyKSk KATnTYMeEFIplpabFh6j5MGit+CBaUEsIjqKGNuFQcHa9THyqMPha2xGoN6tsc1X +X+BYiQuGFWZHMcFuX7vBKlwEYmxILWuoUqY6vPSuHHmrE+8xk4Y60OCpYQHnXo7 DVM34hZI2pqnQT8rr1XtJ8E3MGE8RNNjUtkStT1Y/HiYI1aTTmayiqNxCyunFt1l gphzhwMDdrk7ehcEhaTRQdntnKWTjXL8tCP8XFeyOMca2vC2Z9sD49qJyUjNioXM N/4FOwnHDTu3jLlNqgfZdHtrx26SMD8VXYQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t= 1694605543; x=1694691943; bh=lBicXkbl7p6iD8AYhNBdzSBYrbBSMa+/z2j JChdqbe0=; b=N79rguFdXooFVetMvs9VaRVtqZhT1D3M7VCPGEzxp9iLqewZqgu 1HmWB9qjIBXlfOvaSKsymcMCfPt6cEaRTAeZA8FnXxz2VsNKcWGsnBRbLAJ0DJn2 CfLBz1sSvCCZ6vHfBphbbso3HslHeDekfiNdGMDirJ8ojnsx0CMX8Zpm7Nh5c0vV gbeyKtzzgw4vPzGmaUKV1sJKCmQYKXxlf+qPhJ/M/wM9J34wTq+43hLaXy5ULVh/ dalEai2xyQa23WVKsTZXAUi3d2eSgOOl3TPgIw9cXk6srBTwzZj2691sVGN0At8M 9dMKLPHmI7UI4W0JNqG8ZMpn2HK8ztgBUXQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudeikedggeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofgjfhggtgfgsehtkeertdertdejnecuhfhrohhmpefvhhho mhgrshcuofhonhhjrghlohhnuceothhhohhmrghssehmohhnjhgrlhhonhdrnhgvtheqne cuggftrfgrthhtvghrnhephfejtdejheeuffettdetgefgueetueelgeegffejhedutdek vdeuvdeujefhffeknecuffhomhgrihhnpehthhhrvggrugdrrhhunhdpthhhrhgvrggurd hiugenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpeht hhhomhgrshesmhhonhhjrghlohhnrdhnvght X-ME-Proxy: Feedback-ID: i47234305:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 13 Sep 2023 07:45:38 -0400 (EDT) From: Thomas Monjalon To: dev@dpdk.org Cc: Tyler Retzlaff , David Marchand , Ferruh Yigit , =?UTF-8?q?Morten=20Br=C3=B8rup?= , Andrew Rybchenko , Nithin Dabilpuram , Kiran Kumar K , Sunil Kumar Kori , Satha Rao , Chengwen Feng , Kevin Laatz , Bruce Richardson , Abdullah Sevincer , Shepard Siegel , Ed Czeck , John Miller , Jingjing Wu , Beilei Xing , Qiming Yang , Qi Zhang , Rosen Xu , Wenjun Wu , Matan Azrad , Viacheslav Ovsiienko , Ori Kam , Suanming Mou , Jiawen Wu , Jian Wang , Tianfei Zhang , Xiao Wang , Vijay Kumar Srivastava Subject: [PATCH v3 07/11] drivers: convert to internal control threads Date: Wed, 13 Sep 2023 13:28:20 +0200 Message-ID: <20230913114351.1940459-8-thomas@monjalon.net> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230913114351.1940459-1-thomas@monjalon.net> References: <20230906162226.1618088-1-thomas@monjalon.net> <20230913114351.1940459-1-thomas@monjalon.net> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Calls to rte_ctrl_thread_create() are replaced with rte_thread_create_internal_control(). Other pthread-related functions are replaced with the rte_thread API. Only pthread_cancel() has no replacement. The mlx5 vDPA control threads were real-time threads with an affinity on the core specified by the devarg "event_core". As there is no heavy performance requirement, these threads are converted into real control threads. Some threads in dlb2 and mlx5 are not control threads because they are designed to run on some specific cores. It would be interesting to rework them in future. Signed-off-by: Thomas Monjalon Acked-by: Morten Brørup Acked-by: Tyler Retzlaff Acked-by: Andrew Rybchenko --- drivers/common/cnxk/roc_dev.c | 16 ++-- drivers/common/cnxk/roc_dev_priv.h | 2 +- drivers/common/cnxk/roc_nix_inl_dev.c | 9 +-- drivers/common/cnxk/roc_nix_inl_priv.h | 2 +- drivers/common/cnxk/roc_platform.h | 3 +- drivers/dma/skeleton/skeleton_dmadev.c | 7 +- drivers/event/dlb2/pf/base/dlb2_osdep.h | 14 ++-- drivers/event/dlb2/pf/base/dlb2_resource.c | 39 +++++----- drivers/event/dlb2/pf/dlb2_main.h | 1 - drivers/net/ark/ark_ethdev.c | 11 ++- drivers/net/ark/ark_pktgen.c | 8 +- drivers/net/ark/ark_pktgen.h | 2 +- drivers/net/iavf/iavf_vchnl.c | 2 +- drivers/net/ice/ice_dcf_parent.c | 7 +- drivers/net/ipn3ke/ipn3ke_representor.c | 17 ++--- drivers/net/ixgbe/ixgbe_ethdev.c | 7 +- drivers/net/mlx5/mlx5.h | 2 +- drivers/net/mlx5/mlx5_hws_cnt.c | 42 ++++++----- drivers/net/sfc/sfc_mae.h | 2 +- drivers/net/sfc/sfc_mae_counter.c | 11 ++- drivers/net/txgbe/txgbe_ethdev.c | 6 +- drivers/raw/ifpga/ifpga_rawdev.c | 19 +++-- drivers/vdpa/ifc/ifcvf_vdpa.c | 86 ++++++++++------------ drivers/vdpa/mlx5/mlx5_vdpa.c | 2 +- drivers/vdpa/mlx5/mlx5_vdpa.h | 10 +-- drivers/vdpa/mlx5/mlx5_vdpa_cthread.c | 70 +++++------------- drivers/vdpa/mlx5/mlx5_vdpa_event.c | 64 +++++----------- drivers/vdpa/sfc/sfc_vdpa_ops.c | 15 ++-- drivers/vdpa/sfc/sfc_vdpa_ops.h | 3 +- 29 files changed, 203 insertions(+), 276 deletions(-) diff --git a/drivers/common/cnxk/roc_dev.c b/drivers/common/cnxk/roc_dev.c index 13d3eece47..08fd13f741 100644 --- a/drivers/common/cnxk/roc_dev.c +++ b/drivers/common/cnxk/roc_dev.c @@ -18,7 +18,7 @@ #define ROC_PCI_SRIOV_TOTAL_VF 0x0e /* Total VFs */ /* VF Mbox handler thread name */ -#define MBOX_HANDLER_NAME_MAX_LEN 25 +#define MBOX_HANDLER_NAME_MAX_LEN RTE_THREAD_INTERNAL_NAME_SIZE /* VF interrupt message pending bits - mbox or flr */ #define ROC_DEV_MBOX_PEND BIT_ULL(0) @@ -1070,7 +1070,7 @@ vf_flr_handle_msg(void *param, dev_intr_t *flr) } } -static void * +static uint32_t pf_vf_mbox_thread_main(void *arg) { struct dev *dev = arg; @@ -1114,7 +1114,7 @@ pf_vf_mbox_thread_main(void *arg) pthread_mutex_unlock(&dev->sync.mutex); - return NULL; + return 0; } static void @@ -1455,10 +1455,10 @@ dev_init(struct dev *dev, struct plt_pci_device *pci_dev) pthread_cond_init(&dev->sync.pfvf_msg_cond, NULL); pthread_mutex_init(&dev->sync.mutex, NULL); - snprintf(name, MBOX_HANDLER_NAME_MAX_LEN, "pf%d_vf_msg_hndlr", dev->pf); + snprintf(name, MBOX_HANDLER_NAME_MAX_LEN, "mbox_pf%d", dev->pf); dev->sync.start_thread = true; - rc = plt_ctrl_thread_create(&dev->sync.pfvf_msg_thread, name, NULL, - pf_vf_mbox_thread_main, dev); + rc = plt_thread_create_control(&dev->sync.pfvf_msg_thread, name, + pf_vf_mbox_thread_main, dev); if (rc != 0) { plt_err("Failed to create thread for VF mbox handling\n"); goto iounmap; @@ -1488,7 +1488,7 @@ dev_init(struct dev *dev, struct plt_pci_device *pci_dev) if (dev->sync.start_thread) { dev->sync.start_thread = false; pthread_cond_signal(&dev->sync.pfvf_msg_cond); - pthread_join(dev->sync.pfvf_msg_thread, NULL); + plt_thread_join(dev->sync.pfvf_msg_thread, NULL); pthread_mutex_destroy(&dev->sync.mutex); pthread_cond_destroy(&dev->sync.pfvf_msg_cond); } @@ -1519,7 +1519,7 @@ dev_fini(struct dev *dev, struct plt_pci_device *pci_dev) if (dev->sync.start_thread) { dev->sync.start_thread = false; pthread_cond_signal(&dev->sync.pfvf_msg_cond); - pthread_join(dev->sync.pfvf_msg_thread, NULL); + plt_thread_join(dev->sync.pfvf_msg_thread, NULL); pthread_mutex_destroy(&dev->sync.mutex); pthread_cond_destroy(&dev->sync.pfvf_msg_cond); } diff --git a/drivers/common/cnxk/roc_dev_priv.h b/drivers/common/cnxk/roc_dev_priv.h index 1f84f74ff3..c1a37aa4f0 100644 --- a/drivers/common/cnxk/roc_dev_priv.h +++ b/drivers/common/cnxk/roc_dev_priv.h @@ -73,7 +73,7 @@ dev_is_afvf(uint16_t pf_func) struct mbox_sync { bool start_thread; uint8_t msg_avail; - pthread_t pfvf_msg_thread; + rte_thread_t pfvf_msg_thread; pthread_cond_t pfvf_msg_cond; pthread_mutex_t mutex; }; diff --git a/drivers/common/cnxk/roc_nix_inl_dev.c b/drivers/common/cnxk/roc_nix_inl_dev.c index d76158e30d..7400069309 100644 --- a/drivers/common/cnxk/roc_nix_inl_dev.c +++ b/drivers/common/cnxk/roc_nix_inl_dev.c @@ -746,7 +746,7 @@ inl_outb_soft_exp_poll(struct nix_inl_dev *inl_dev, uint32_t ring_idx) } } -static void * +static uint32_t nix_inl_outb_poll_thread(void *args) { struct nix_inl_dev *inl_dev = args; @@ -816,9 +816,8 @@ nix_inl_outb_poll_thread_setup(struct nix_inl_dev *inl_dev) soft_exp_consumer_cnt = 0; soft_exp_poll_thread_exit = false; - rc = plt_ctrl_thread_create(&inl_dev->soft_exp_poll_thread, - "OUTB_SOFT_EXP_POLL_THREAD", NULL, - nix_inl_outb_poll_thread, inl_dev); + rc = plt_thread_create_control(&inl_dev->soft_exp_poll_thread, + "outb-poll", nix_inl_outb_poll_thread, inl_dev); if (rc) { plt_bitmap_free(inl_dev->soft_exp_ring_bmap); plt_free(inl_dev->soft_exp_ring_bmap_mem); @@ -1020,7 +1019,7 @@ roc_nix_inl_dev_fini(struct roc_nix_inl_dev *roc_inl_dev) if (inl_dev->set_soft_exp_poll) { soft_exp_poll_thread_exit = true; - pthread_join(inl_dev->soft_exp_poll_thread, NULL); + rte_thread_join(inl_dev->soft_exp_poll_thread, NULL); plt_bitmap_free(inl_dev->soft_exp_ring_bmap); plt_free(inl_dev->soft_exp_ring_bmap_mem); plt_free(inl_dev->sa_soft_exp_ring); diff --git a/drivers/common/cnxk/roc_nix_inl_priv.h b/drivers/common/cnxk/roc_nix_inl_priv.h index b0a8976c6b..b2b89227b1 100644 --- a/drivers/common/cnxk/roc_nix_inl_priv.h +++ b/drivers/common/cnxk/roc_nix_inl_priv.h @@ -67,7 +67,7 @@ struct nix_inl_dev { struct roc_cpt_lf cpt_lf; /* OUTB soft expiry poll thread */ - pthread_t soft_exp_poll_thread; + rte_thread_t soft_exp_poll_thread; uint32_t soft_exp_poll_freq; uint64_t *sa_soft_exp_ring; bool set_soft_exp_poll; diff --git a/drivers/common/cnxk/roc_platform.h b/drivers/common/cnxk/roc_platform.h index 08f83aba12..d8f2707a6e 100644 --- a/drivers/common/cnxk/roc_platform.h +++ b/drivers/common/cnxk/roc_platform.h @@ -134,7 +134,8 @@ #define plt_intr_disable rte_intr_disable #define plt_thread_is_intr rte_thread_is_intr #define plt_intr_callback_fn rte_intr_callback_fn -#define plt_ctrl_thread_create rte_ctrl_thread_create +#define plt_thread_create_control rte_thread_create_internal_control +#define plt_thread_join rte_thread_join #define plt_intr_efd_counter_size_get rte_intr_efd_counter_size_get #define plt_intr_efd_counter_size_set rte_intr_efd_counter_size_set diff --git a/drivers/dma/skeleton/skeleton_dmadev.c b/drivers/dma/skeleton/skeleton_dmadev.c index 4fc2319668..eabe88fada 100644 --- a/drivers/dma/skeleton/skeleton_dmadev.c +++ b/drivers/dma/skeleton/skeleton_dmadev.c @@ -100,7 +100,7 @@ static int skeldma_start(struct rte_dma_dev *dev) { struct skeldma_hw *hw = dev->data->dev_private; - char name[RTE_MAX_THREAD_NAME_LEN]; + char name[RTE_THREAD_INTERNAL_NAME_SIZE]; rte_cpuset_t cpuset; int ret; @@ -127,8 +127,9 @@ skeldma_start(struct rte_dma_dev *dev) rte_mb(); - snprintf(name, sizeof(name), "dpdk-dma-skel%d", dev->data->dev_id); - ret = rte_thread_create_control(&hw->thread, name, cpucopy_thread, dev); + snprintf(name, sizeof(name), "dma-skel%d", dev->data->dev_id); + ret = rte_thread_create_internal_control(&hw->thread, name, + cpucopy_thread, dev); if (ret) { SKELDMA_LOG(ERR, "Start cpucopy thread fail!"); return -EINVAL; diff --git a/drivers/event/dlb2/pf/base/dlb2_osdep.h b/drivers/event/dlb2/pf/base/dlb2_osdep.h index 75860cca4d..06d69f39b1 100644 --- a/drivers/event/dlb2/pf/base/dlb2_osdep.h +++ b/drivers/event/dlb2/pf/base/dlb2_osdep.h @@ -8,7 +8,6 @@ #include #include #include -#include #include #include @@ -154,7 +153,7 @@ static inline void os_fence_hcw(struct dlb2_hw *hw, u64 *pp_addr) * map and unmap requests. To prevent deadlock, this function gives other * threads a chance to grab the resource mutex and configure hardware. */ -static void *dlb2_complete_queue_map_unmap(void *__args) +static uint32_t dlb2_complete_queue_map_unmap(void *__args) { struct dlb2_dev *dlb2_dev = (struct dlb2_dev *)__args; int ret; @@ -180,7 +179,7 @@ static void *dlb2_complete_queue_map_unmap(void *__args) rte_spinlock_unlock(&dlb2_dev->resource_mutex); - return NULL; + return 0; } @@ -194,16 +193,13 @@ static void *dlb2_complete_queue_map_unmap(void *__args) static inline void os_schedule_work(struct dlb2_hw *hw) { struct dlb2_dev *dlb2_dev; - pthread_t complete_queue_map_unmap_thread; + rte_thread_t complete_queue_map_unmap_thread; int ret; dlb2_dev = container_of(hw, struct dlb2_dev, hw); - ret = rte_ctrl_thread_create(&complete_queue_map_unmap_thread, - "dpdk-dlb-qunmap", - NULL, - dlb2_complete_queue_map_unmap, - dlb2_dev); + ret = rte_thread_create_internal_control(&complete_queue_map_unmap_thread, + "dlb-qunmap", dlb2_complete_queue_map_unmap, dlb2_dev); if (ret) DLB2_ERR(dlb2_dev, "Could not create queue complete map/unmap thread, err=%d\n", diff --git a/drivers/event/dlb2/pf/base/dlb2_resource.c b/drivers/event/dlb2/pf/base/dlb2_resource.c index dd8390a013..7ce3e3531c 100644 --- a/drivers/event/dlb2/pf/base/dlb2_resource.c +++ b/drivers/event/dlb2/pf/base/dlb2_resource.c @@ -759,18 +759,13 @@ static int dlb2_attach_ldb_queues(struct dlb2_hw *hw, } static int -dlb2_pp_profile(struct dlb2_hw *hw, int port, int cpu, bool is_ldb) +dlb2_pp_profile(struct dlb2_hw *hw, int port, bool is_ldb) { u64 cycle_start = 0ULL, cycle_end = 0ULL; struct dlb2_hcw hcw_mem[DLB2_HCW_MEM_SIZE], *hcw; void __iomem *pp_addr; - cpu_set_t cpuset; int i; - CPU_ZERO(&cpuset); - CPU_SET(cpu, &cpuset); - sched_setaffinity(0, sizeof(cpuset), &cpuset); - pp_addr = os_map_producer_port(hw, port, is_ldb); /* Point hcw to a 64B-aligned location */ @@ -797,18 +792,15 @@ dlb2_pp_profile(struct dlb2_hw *hw, int port, int cpu, bool is_ldb) return (int)(cycle_end - cycle_start); } -static void * +static uint32_t dlb2_pp_profile_func(void *data) { struct dlb2_pp_thread_data *thread_data = data; - int cycles; - cycles = dlb2_pp_profile(thread_data->hw, thread_data->pp, - thread_data->cpu, thread_data->is_ldb); + thread_data->cycles = dlb2_pp_profile(thread_data->hw, + thread_data->pp, thread_data->is_ldb); - thread_data->cycles = cycles; - - return NULL; + return 0; } static int dlb2_pp_cycle_comp(const void *a, const void *b) @@ -831,7 +823,9 @@ dlb2_get_pp_allocation(struct dlb2_hw *hw, int cpu, int port_type) int num_ports_per_sort, num_ports, num_sort, i, err; bool is_ldb = (port_type == DLB2_LDB_PORT); int *port_allocations; - pthread_t pthread; + rte_thread_t thread; + rte_thread_attr_t th_attr; + char th_name[RTE_THREAD_INTERNAL_NAME_SIZE]; if (is_ldb) { port_allocations = hw->ldb_pp_allocations; @@ -857,16 +851,25 @@ dlb2_get_pp_allocation(struct dlb2_hw *hw, int cpu, int port_type) dlb2_thread_data[i].pp = i; dlb2_thread_data[i].cycles = 0; dlb2_thread_data[i].hw = hw; - dlb2_thread_data[i].cpu = cpu; - err = pthread_create(&pthread, NULL, &dlb2_pp_profile_func, - &dlb2_thread_data[i]); + err = rte_thread_attr_init(&th_attr); + if (err != 0) { + DLB2_LOG_ERR(": thread attribute failed! err=%d", err); + return; + } + CPU_SET(cpu, &th_attr.cpuset); + + err = rte_thread_create(&thread, &th_attr, + &dlb2_pp_profile_func, &dlb2_thread_data[i]); if (err) { DLB2_LOG_ERR(": thread creation failed! err=%d", err); return; } - err = pthread_join(pthread, NULL); + snprintf(th_name, sizeof(th_name), "dlb2-pp%d", cpu); + rte_thread_set_prefixed_name(thread, th_name); + + err = rte_thread_join(thread, NULL); if (err) { DLB2_LOG_ERR(": thread join failed! err=%d", err); return; diff --git a/drivers/event/dlb2/pf/dlb2_main.h b/drivers/event/dlb2/pf/dlb2_main.h index 4c64d72e9c..12912a2dec 100644 --- a/drivers/event/dlb2/pf/dlb2_main.h +++ b/drivers/event/dlb2/pf/dlb2_main.h @@ -52,7 +52,6 @@ struct dlb2_dev { struct dlb2_pp_thread_data { struct dlb2_hw *hw; int pp; - int cpu; bool is_ldb; int cycles; }; diff --git a/drivers/net/ark/ark_ethdev.c b/drivers/net/ark/ark_ethdev.c index 30814650bf..3ddcc9b461 100644 --- a/drivers/net/ark/ark_ethdev.c +++ b/drivers/net/ark/ark_ethdev.c @@ -598,17 +598,16 @@ eth_ark_dev_start(struct rte_eth_dev *dev) ark_pktchkr_run(ark->pc); if (!ark->isvf && ark->start_pg && !ark->pg_running) { - pthread_t thread; + rte_thread_t thread; /* Delay packet generator start allow the hardware to be ready * This is only used for sanity checking with internal generator */ - char tname[32]; - snprintf(tname, sizeof(tname), "dpdk-ark-pg%d", - dev->data->port_id); + char tname[RTE_THREAD_INTERNAL_NAME_SIZE]; + snprintf(tname, sizeof(tname), "ark-pg%d", dev->data->port_id); - if (rte_ctrl_thread_create(&thread, tname, NULL, - ark_pktgen_delay_start, ark->pg)) { + if (rte_thread_create_internal_control(&thread, tname, + ark_pktgen_delay_start, ark->pg)) { ARK_PMD_LOG(ERR, "Could not create pktgen " "starter thread\n"); return -1; diff --git a/drivers/net/ark/ark_pktgen.c b/drivers/net/ark/ark_pktgen.c index cd2d3adc51..69ff7072b2 100644 --- a/drivers/net/ark/ark_pktgen.c +++ b/drivers/net/ark/ark_pktgen.c @@ -4,10 +4,10 @@ #include #include -#include #include #include +#include #include "ark_pktgen.h" #include "ark_logs.h" @@ -467,7 +467,7 @@ ark_pktgen_setup(ark_pkt_gen_t handle) } } -void * +uint32_t ark_pktgen_delay_start(void *arg) { struct ark_pkt_gen_inst *inst = (struct ark_pkt_gen_inst *)arg; @@ -476,8 +476,8 @@ ark_pktgen_delay_start(void *arg) * perform a blind sleep here to ensure that the external test * application has time to setup the test before we generate packets */ - pthread_detach(pthread_self()); + rte_thread_detach(rte_thread_self()); usleep(100000); ark_pktgen_run(inst); - return NULL; + return 0; } diff --git a/drivers/net/ark/ark_pktgen.h b/drivers/net/ark/ark_pktgen.h index 7147fe1bd4..925058367c 100644 --- a/drivers/net/ark/ark_pktgen.h +++ b/drivers/net/ark/ark_pktgen.h @@ -75,6 +75,6 @@ void ark_pktgen_set_hdr_dW(ark_pkt_gen_t handle, uint32_t *hdr); void ark_pktgen_set_start_offset(ark_pkt_gen_t handle, uint32_t x); void ark_pktgen_parse(char *argv); void ark_pktgen_setup(ark_pkt_gen_t handle); -void *ark_pktgen_delay_start(void *arg); +uint32_t ark_pktgen_delay_start(void *arg); #endif diff --git a/drivers/net/iavf/iavf_vchnl.c b/drivers/net/iavf/iavf_vchnl.c index 6baa6d5955..b3e106b41f 100644 --- a/drivers/net/iavf/iavf_vchnl.c +++ b/drivers/net/iavf/iavf_vchnl.c @@ -135,7 +135,7 @@ iavf_dev_event_handler_init(void) TAILQ_INIT(&handler->pending); pthread_mutex_init(&handler->lock, NULL); - if (rte_thread_create_control(&handler->tid, "dpdk-iavf-event", + if (rte_thread_create_internal_control(&handler->tid, "iavf-event", iavf_dev_event_handle, NULL)) { __atomic_fetch_sub(&handler->ndev, 1, __ATOMIC_RELAXED); return -1; diff --git a/drivers/net/ice/ice_dcf_parent.c b/drivers/net/ice/ice_dcf_parent.c index d3bb65874f..173ed9f81d 100644 --- a/drivers/net/ice/ice_dcf_parent.c +++ b/drivers/net/ice/ice_dcf_parent.c @@ -165,9 +165,8 @@ ice_dcf_vsi_update_service_handler(void *param) static void start_vsi_reset_thread(struct ice_dcf_hw *dcf_hw, bool vfr, uint16_t vf_id) { -#define THREAD_NAME_LEN 16 struct ice_dcf_reset_event_param *param; - char name[THREAD_NAME_LEN]; + char name[RTE_THREAD_INTERNAL_NAME_SIZE]; rte_thread_t thread; int ret; @@ -181,8 +180,8 @@ start_vsi_reset_thread(struct ice_dcf_hw *dcf_hw, bool vfr, uint16_t vf_id) param->vfr = vfr; param->vf_id = vf_id; - snprintf(name, sizeof(name), "dpdk-ice-rst%u", vf_id); - ret = rte_thread_create_control(&thread, name, + snprintf(name, sizeof(name), "ice-rst%u", vf_id); + ret = rte_thread_create_internal_control(&thread, name, ice_dcf_vsi_update_service_handler, param); if (ret != 0) { PMD_DRV_LOG(ERR, "Failed to start the thread for reset handling"); diff --git a/drivers/net/ipn3ke/ipn3ke_representor.c b/drivers/net/ipn3ke/ipn3ke_representor.c index c82f8b5333..57ec7a19bc 100644 --- a/drivers/net/ipn3ke/ipn3ke_representor.c +++ b/drivers/net/ipn3ke/ipn3ke_representor.c @@ -27,7 +27,7 @@ #include "ipn3ke_ethdev.h" static int ipn3ke_rpst_scan_num; -static pthread_t ipn3ke_rpst_scan_thread; +static rte_thread_t ipn3ke_rpst_scan_thread; /** Double linked list of representor port. */ TAILQ_HEAD(ipn3ke_rpst_list, ipn3ke_rpst); @@ -2558,7 +2558,7 @@ ipn3ke_rpst_link_check(struct ipn3ke_rpst *rpst) return 0; } -static void * +static uint32_t ipn3ke_rpst_scan_handle_request(__rte_unused void *param) { struct ipn3ke_rpst *rpst; @@ -2580,10 +2580,10 @@ ipn3ke_rpst_scan_handle_request(__rte_unused void *param) rte_delay_us(50 * MS); if (num == 0 || num == 0xffffff) - return NULL; + return 0; } - return NULL; + return 0; } static int @@ -2592,20 +2592,19 @@ ipn3ke_rpst_scan_check(void) int ret; if (ipn3ke_rpst_scan_num == 1) { - ret = rte_ctrl_thread_create(&ipn3ke_rpst_scan_thread, - "dpdk-ipn3ke-scn", - NULL, + ret = rte_thread_create_internal_control(&ipn3ke_rpst_scan_thread, + "ipn3ke-scn", ipn3ke_rpst_scan_handle_request, NULL); if (ret) { IPN3KE_AFU_PMD_ERR("Fail to create ipn3ke rpst scan thread"); return -1; } } else if (ipn3ke_rpst_scan_num == 0) { - ret = pthread_cancel(ipn3ke_rpst_scan_thread); + ret = pthread_cancel((pthread_t)ipn3ke_rpst_scan_thread.opaque_id); if (ret) IPN3KE_AFU_PMD_ERR("Can't cancel the thread"); - ret = pthread_join(ipn3ke_rpst_scan_thread, NULL); + ret = rte_thread_join(ipn3ke_rpst_scan_thread, NULL); if (ret) IPN3KE_AFU_PMD_ERR("Can't join the thread"); diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c index 64e7e5d7d6..cd4a85b3a7 100644 --- a/drivers/net/ixgbe/ixgbe_ethdev.c +++ b/drivers/net/ixgbe/ixgbe_ethdev.c @@ -4328,10 +4328,9 @@ ixgbe_dev_link_update_share(struct rte_eth_dev *dev, * when there is no link thread running. */ intr->flags |= IXGBE_FLAG_NEED_LINK_CONFIG; - if (rte_thread_create_control(&ad->link_thread_tid, - "dpdk-ixgbe-link", - ixgbe_dev_setup_link_thread_handler, - dev) < 0) { + if (rte_thread_create_internal_control(&ad->link_thread_tid, + "ixgbe-link", + ixgbe_dev_setup_link_thread_handler, dev) < 0) { PMD_DRV_LOG(ERR, "Create link thread failed!"); /* NOTE: review for potential ordering optimization */ diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 3785103308..c587e13c63 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -1368,7 +1368,7 @@ struct mlx5_hws_cnt_svc_mng { uint32_t refcnt; uint32_t service_core; uint32_t query_interval; - pthread_t service_thread; + rte_thread_t service_thread; uint8_t svc_running; struct mlx5_hws_aso_mng aso_mng __rte_cache_aligned; }; diff --git a/drivers/net/mlx5/mlx5_hws_cnt.c b/drivers/net/mlx5/mlx5_hws_cnt.c index a0f0febbc7..f556a9fbcc 100644 --- a/drivers/net/mlx5/mlx5_hws_cnt.c +++ b/drivers/net/mlx5/mlx5_hws_cnt.c @@ -9,6 +9,7 @@ #include #include #include +#include #if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H) @@ -286,7 +287,7 @@ mlx5_hws_cnt_raw_data_alloc(struct mlx5_dev_ctx_shared *sh, uint32_t n) return NULL; } -static void * +static uint32_t mlx5_hws_cnt_svc(void *opaque) { struct mlx5_dev_ctx_shared *sh = @@ -318,7 +319,7 @@ mlx5_hws_cnt_svc(void *opaque) if (interval > query_us) rte_delay_us_sleep(sleep_us); } - return NULL; + return 0; } static void @@ -438,36 +439,37 @@ mlx5_hws_cnt_pool_init(struct mlx5_dev_ctx_shared *sh, int mlx5_hws_cnt_service_thread_create(struct mlx5_dev_ctx_shared *sh) { - char name[RTE_MAX_THREAD_NAME_LEN]; - rte_cpuset_t cpuset; + char name[RTE_THREAD_INTERNAL_NAME_SIZE]; + rte_thread_attr_t attr; int ret; uint32_t service_core = sh->cnt_svc->service_core; - CPU_ZERO(&cpuset); + ret = rte_thread_attr_init(&attr); + if (ret != 0) + goto error; + CPU_SET(service_core, &attr.cpuset); sh->cnt_svc->svc_running = 1; - ret = pthread_create(&sh->cnt_svc->service_thread, NULL, - mlx5_hws_cnt_svc, sh); - if (ret != 0) { - DRV_LOG(ERR, "Failed to create HW steering's counter service thread."); - return -ENOSYS; - } - snprintf(name, RTE_MAX_THREAD_NAME_LEN, "dpdk-mlx5-%d", service_core); - rte_thread_set_name((rte_thread_t){(uintptr_t)sh->cnt_svc->service_thread}, - name); - CPU_SET(service_core, &cpuset); - pthread_setaffinity_np(sh->cnt_svc->service_thread, sizeof(cpuset), - &cpuset); + ret = rte_thread_create(&sh->cnt_svc->service_thread, + &attr, mlx5_hws_cnt_svc, sh); + if (ret != 0) + goto error; + snprintf(name, sizeof(name), "mlx5-cn%d", service_core); + rte_thread_set_prefixed_name(sh->cnt_svc->service_thread, name); + return 0; +error: + DRV_LOG(ERR, "Failed to create HW steering's counter service thread."); + return ret; } void mlx5_hws_cnt_service_thread_destroy(struct mlx5_dev_ctx_shared *sh) { - if (sh->cnt_svc->service_thread == 0) + if (sh->cnt_svc->service_thread.opaque_id == 0) return; sh->cnt_svc->svc_running = 0; - pthread_join(sh->cnt_svc->service_thread, NULL); - sh->cnt_svc->service_thread = 0; + rte_thread_join(sh->cnt_svc->service_thread, NULL); + sh->cnt_svc->service_thread.opaque_id = 0; } static int diff --git a/drivers/net/sfc/sfc_mae.h b/drivers/net/sfc/sfc_mae.h index 646d055ac1..7988424449 100644 --- a/drivers/net/sfc/sfc_mae.h +++ b/drivers/net/sfc/sfc_mae.h @@ -196,7 +196,7 @@ struct sfc_mae_counter_registry { } service; struct { /** Counter thread ID */ - pthread_t id; + rte_thread_t id; /** The thread should keep running */ bool run; } thread; diff --git a/drivers/net/sfc/sfc_mae_counter.c b/drivers/net/sfc/sfc_mae_counter.c index 631b54afeb..ba17295719 100644 --- a/drivers/net/sfc/sfc_mae_counter.c +++ b/drivers/net/sfc/sfc_mae_counter.c @@ -490,7 +490,7 @@ sfc_mae_counter_service_routine(void *arg) return 0; } -static void * +static uint32_t sfc_mae_counter_thread(void *data) { struct sfc_adapter *sa = data; @@ -521,7 +521,7 @@ sfc_mae_counter_thread(void *data) } } - return NULL; + return 0; } static void @@ -687,7 +687,7 @@ sfc_mae_counter_thread_stop(struct sfc_adapter *sa) __atomic_store_n(&counter_registry->polling.thread.run, false, __ATOMIC_RELEASE); - rc = pthread_join(counter_registry->polling.thread.id, NULL); + rc = rte_thread_join(counter_registry->polling.thread.id, NULL); if (rc != 0) sfc_err(sa, "failed to join the MAE counter polling thread"); @@ -710,9 +710,8 @@ sfc_mae_counter_thread_spawn(struct sfc_adapter *sa, counter_registry->polling_mode = SFC_MAE_COUNTER_POLLING_THREAD; counter_registry->polling.thread.run = true; - rc = rte_ctrl_thread_create(&sa->mae.counter_registry.polling.thread.id, - "dpdk-sfc-maecnt", NULL, - sfc_mae_counter_thread, sa); + rc = rte_thread_create_internal_control(&sa->mae.counter_registry.polling.thread.id, + "sfc-maecnt", sfc_mae_counter_thread, sa); return rc; } diff --git a/drivers/net/txgbe/txgbe_ethdev.c b/drivers/net/txgbe/txgbe_ethdev.c index 198fac8318..15911a3621 100644 --- a/drivers/net/txgbe/txgbe_ethdev.c +++ b/drivers/net/txgbe/txgbe_ethdev.c @@ -2896,9 +2896,9 @@ txgbe_dev_link_update_share(struct rte_eth_dev *dev, * when there is no link thread running. */ intr->flags |= TXGBE_FLAG_NEED_LINK_CONFIG; - if (rte_thread_create_control(&ad->link_thread_tid, - "dpdk-txgbe-link", - txgbe_dev_setup_link_thread_handler, dev) < 0) { + if (rte_thread_create_internal_control(&ad->link_thread_tid, + "txgbe-link", + txgbe_dev_setup_link_thread_handler, dev) < 0) { PMD_DRV_LOG(ERR, "Create link thread failed!"); __atomic_clear(&ad->link_thread_running, __ATOMIC_SEQ_CST); } diff --git a/drivers/raw/ifpga/ifpga_rawdev.c b/drivers/raw/ifpga/ifpga_rawdev.c index a7b18299de..f89bd3f9e2 100644 --- a/drivers/raw/ifpga/ifpga_rawdev.c +++ b/drivers/raw/ifpga/ifpga_rawdev.c @@ -74,7 +74,7 @@ static const struct rte_pci_id pci_ifpga_map[] = { static struct ifpga_rawdev ifpga_rawdevices[IFPGA_RAWDEV_NUM]; static int ifpga_monitor_refcnt; -static pthread_t ifpga_monitor_start_thread; +static rte_thread_t ifpga_monitor_start_thread; static struct ifpga_rawdev * ifpga_rawdev_allocate(struct rte_rawdev *rawdev); @@ -504,7 +504,7 @@ static int set_surprise_link_check_aer( return -EFAULT; } -static void * +static uint32_t ifpga_rawdev_gsd_handle(__rte_unused void *param) { struct ifpga_rawdev *ifpga_rdev; @@ -532,7 +532,7 @@ ifpga_rawdev_gsd_handle(__rte_unused void *param) rte_delay_us(100 * MS); } - return NULL; + return 0; } static int @@ -550,11 +550,10 @@ ifpga_monitor_start_func(struct ifpga_rawdev *dev) dev->poll_enabled = 1; if (!__atomic_fetch_add(&ifpga_monitor_refcnt, 1, __ATOMIC_RELAXED)) { - ret = rte_ctrl_thread_create(&ifpga_monitor_start_thread, - "dpdk-ifpga-mon", NULL, - ifpga_rawdev_gsd_handle, NULL); + ret = rte_thread_create_internal_control(&ifpga_monitor_start_thread, + "ifpga-mon", ifpga_rawdev_gsd_handle, NULL); if (ret != 0) { - ifpga_monitor_start_thread = 0; + ifpga_monitor_start_thread.opaque_id = 0; IFPGA_RAWDEV_PMD_ERR( "Fail to create ifpga monitor thread"); return -1; @@ -575,12 +574,12 @@ ifpga_monitor_stop_func(struct ifpga_rawdev *dev) dev->poll_enabled = 0; if (!(__atomic_fetch_sub(&ifpga_monitor_refcnt, 1, __ATOMIC_RELAXED) - 1) && - ifpga_monitor_start_thread) { - ret = pthread_cancel(ifpga_monitor_start_thread); + ifpga_monitor_start_thread.opaque_id != 0) { + ret = pthread_cancel((pthread_t)ifpga_monitor_start_thread.opaque_id); if (ret) IFPGA_RAWDEV_PMD_ERR("Can't cancel the thread"); - ret = pthread_join(ifpga_monitor_start_thread, NULL); + ret = rte_thread_join(ifpga_monitor_start_thread, NULL); if (ret) IFPGA_RAWDEV_PMD_ERR("Can't join the thread"); diff --git a/drivers/vdpa/ifc/ifcvf_vdpa.c b/drivers/vdpa/ifc/ifcvf_vdpa.c index 55346bbc08..f034bd59ba 100644 --- a/drivers/vdpa/ifc/ifcvf_vdpa.c +++ b/drivers/vdpa/ifc/ifcvf_vdpa.c @@ -58,8 +58,8 @@ struct ifcvf_internal { int vfio_container_fd; int vfio_group_fd; int vfio_dev_fd; - pthread_t tid; /* thread for notify relay */ - pthread_t intr_tid; /* thread for config space change interrupt relay */ + rte_thread_t tid; /* thread for notify relay */ + rte_thread_t intr_tid; /* thread for config space change interrupt relay */ int epfd; int csc_epfd; int vid; @@ -496,7 +496,7 @@ vdpa_disable_vfio_intr(struct ifcvf_internal *internal) return 0; } -static void * +static uint32_t notify_relay(void *arg) { int i, kickfd, epfd, nfds = 0; @@ -514,7 +514,7 @@ notify_relay(void *arg) epfd = epoll_create(IFCVF_MAX_QUEUES * 2); if (epfd < 0) { DRV_LOG(ERR, "failed to create epoll instance."); - return NULL; + return 1; } internal->epfd = epfd; @@ -527,7 +527,7 @@ notify_relay(void *arg) ev.data.u64 = qid | (uint64_t)vring.kickfd << 32; if (epoll_ctl(epfd, EPOLL_CTL_ADD, vring.kickfd, &ev) < 0) { DRV_LOG(ERR, "epoll add error: %s", strerror(errno)); - return NULL; + return 1; } } @@ -537,7 +537,7 @@ notify_relay(void *arg) if (errno == EINTR) continue; DRV_LOG(ERR, "epoll_wait return fail\n"); - return NULL; + return 1; } for (i = 0; i < nfds; i++) { @@ -561,18 +561,18 @@ notify_relay(void *arg) } } - return NULL; + return 0; } static int setup_notify_relay(struct ifcvf_internal *internal) { - char name[THREAD_NAME_LEN]; + char name[RTE_THREAD_INTERNAL_NAME_SIZE]; int ret; - snprintf(name, sizeof(name), "dpdk-ifc-noti%d", internal->vid); - ret = rte_ctrl_thread_create(&internal->tid, name, NULL, notify_relay, - (void *)internal); + snprintf(name, sizeof(name), "ifc-noti%d", internal->vid); + ret = rte_thread_create_internal_control(&internal->tid, name, + notify_relay, internal); if (ret != 0) { DRV_LOG(ERR, "failed to create notify relay pthread."); return -1; @@ -584,13 +584,11 @@ setup_notify_relay(struct ifcvf_internal *internal) static int unset_notify_relay(struct ifcvf_internal *internal) { - void *status; - - if (internal->tid) { - pthread_cancel(internal->tid); - pthread_join(internal->tid, &status); + if (internal->tid.opaque_id != 0) { + pthread_cancel((pthread_t)internal->tid.opaque_id); + rte_thread_join(internal->tid, NULL); } - internal->tid = 0; + internal->tid.opaque_id = 0; if (internal->epfd >= 0) close(internal->epfd); @@ -610,7 +608,7 @@ virtio_interrupt_handler(struct ifcvf_internal *internal) DRV_LOG(ERR, "failed to notify the guest about configuration space change."); } -static void * +static uint32_t intr_relay(void *arg) { struct ifcvf_internal *internal = (struct ifcvf_internal *)arg; @@ -623,7 +621,7 @@ intr_relay(void *arg) csc_epfd = epoll_create(1); if (csc_epfd < 0) { DRV_LOG(ERR, "failed to create epoll for config space change."); - return NULL; + return 1; } ev.events = EPOLLIN | EPOLLPRI | EPOLLRDHUP | EPOLLHUP; @@ -672,18 +670,18 @@ intr_relay(void *arg) close(csc_epfd); internal->csc_epfd = -1; - return NULL; + return 0; } static int setup_intr_relay(struct ifcvf_internal *internal) { - char name[THREAD_NAME_LEN]; + char name[RTE_THREAD_INTERNAL_NAME_SIZE]; int ret; - snprintf(name, sizeof(name), "dpdk-ifc-int%d", internal->vid); - ret = rte_ctrl_thread_create(&internal->intr_tid, name, NULL, - intr_relay, (void *)internal); + snprintf(name, sizeof(name), "ifc-int%d", internal->vid); + ret = rte_thread_create_internal_control(&internal->intr_tid, name, + intr_relay, (void *)internal); if (ret) { DRV_LOG(ERR, "failed to create notify relay pthread."); return -1; @@ -694,13 +692,11 @@ setup_intr_relay(struct ifcvf_internal *internal) static void unset_intr_relay(struct ifcvf_internal *internal) { - void *status; - - if (internal->intr_tid) { - pthread_cancel(internal->intr_tid); - pthread_join(internal->intr_tid, &status); + if (internal->intr_tid.opaque_id != 0) { + pthread_cancel((pthread_t)internal->intr_tid.opaque_id); + rte_thread_join(internal->intr_tid, NULL); } - internal->intr_tid = 0; + internal->intr_tid.opaque_id = 0; if (internal->csc_epfd >= 0) close(internal->csc_epfd); @@ -922,7 +918,7 @@ update_used_ring(struct ifcvf_internal *internal, uint16_t qid) rte_vhost_vring_call(internal->vid, qid); } -static void * +static uint32_t vring_relay(void *arg) { int i, vid, epfd, fd, nfds; @@ -941,7 +937,7 @@ vring_relay(void *arg) epfd = epoll_create(IFCVF_MAX_QUEUES * 2); if (epfd < 0) { DRV_LOG(ERR, "failed to create epoll instance."); - return NULL; + return 1; } internal->epfd = epfd; @@ -952,7 +948,7 @@ vring_relay(void *arg) ev.data.u64 = qid << 1 | (uint64_t)vring.kickfd << 32; if (epoll_ctl(epfd, EPOLL_CTL_ADD, vring.kickfd, &ev) < 0) { DRV_LOG(ERR, "epoll add error: %s", strerror(errno)); - return NULL; + return 1; } } @@ -966,7 +962,7 @@ vring_relay(void *arg) if (epoll_ctl(epfd, EPOLL_CTL_ADD, internal->intr_fd[qid], &ev) < 0) { DRV_LOG(ERR, "epoll add error: %s", strerror(errno)); - return NULL; + return 1; } update_used_ring(internal, qid); } @@ -982,7 +978,7 @@ vring_relay(void *arg) if (errno == EINTR) continue; DRV_LOG(ERR, "epoll_wait return fail."); - return NULL; + return 1; } for (i = 0; i < nfds; i++) { @@ -1010,18 +1006,18 @@ vring_relay(void *arg) } } - return NULL; + return 0; } static int setup_vring_relay(struct ifcvf_internal *internal) { - char name[THREAD_NAME_LEN]; + char name[RTE_THREAD_INTERNAL_NAME_SIZE]; int ret; - snprintf(name, sizeof(name), "dpdk-ifc-ring%d", internal->vid); - ret = rte_ctrl_thread_create(&internal->tid, name, NULL, vring_relay, - (void *)internal); + snprintf(name, sizeof(name), "ifc-ring%d", internal->vid); + ret = rte_thread_create_internal_control(&internal->tid, name, + vring_relay, internal); if (ret != 0) { DRV_LOG(ERR, "failed to create ring relay pthread."); return -1; @@ -1033,13 +1029,11 @@ setup_vring_relay(struct ifcvf_internal *internal) static int unset_vring_relay(struct ifcvf_internal *internal) { - void *status; - - if (internal->tid) { - pthread_cancel(internal->tid); - pthread_join(internal->tid, &status); + if (internal->tid.opaque_id != 0) { + pthread_cancel((pthread_t)internal->tid.opaque_id); + rte_thread_join(internal->tid, NULL); } - internal->tid = 0; + internal->tid.opaque_id = 0; if (internal->epfd >= 0) close(internal->epfd); diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.c b/drivers/vdpa/mlx5/mlx5_vdpa.c index f1737f82a8..01a5877d08 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa.c @@ -844,7 +844,7 @@ mlx5_vdpa_dev_probe(struct mlx5_common_device *cdev, mlx5_vdpa_config_get(mkvlist, priv); if (priv->use_c_thread) { if (conf_thread_mng.initializer_priv == priv) - if (mlx5_vdpa_mult_threads_create(priv->event_core)) + if (mlx5_vdpa_mult_threads_create()) goto error; __atomic_fetch_add(&conf_thread_mng.refcnt, 1, __ATOMIC_RELAXED); diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.h b/drivers/vdpa/mlx5/mlx5_vdpa.h index dc4dfba5ed..7b37c98e74 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/mlx5_vdpa.h @@ -19,6 +19,7 @@ #endif #include #include +#include #include #include @@ -99,7 +100,7 @@ struct mlx5_vdpa_task { /* Generic mlx5_vdpa_c_thread information. */ struct mlx5_vdpa_c_thread { - pthread_t tid; + rte_thread_t tid; struct rte_ring *rng; pthread_cond_t c_cond; }; @@ -182,7 +183,7 @@ struct mlx5_vdpa_priv { rte_spinlock_t db_lock; pthread_mutex_t steer_update_lock; uint64_t no_traffic_counter; - pthread_t timer_tid; + rte_thread_t timer_tid; int event_mode; int event_core; /* Event thread cpu affinity core. */ uint32_t event_us; @@ -563,14 +564,11 @@ mlx5_vdpa_is_modify_virtq_supported(struct mlx5_vdpa_priv *priv); /** * Create configuration multi-threads resource * - * @param[in] cpu_core - * CPU core number to set configuration threads affinity to. - * * @return * 0 on success, a negative value otherwise. */ int -mlx5_vdpa_mult_threads_create(int cpu_core); +mlx5_vdpa_mult_threads_create(void); /** * Destroy configuration multi-threads resource diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_cthread.c b/drivers/vdpa/mlx5/mlx5_vdpa_cthread.c index a1d291bf71..1a5fd3eb17 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa_cthread.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa_cthread.c @@ -96,11 +96,10 @@ mlx5_vdpa_c_thread_wait_bulk_tasks_done(uint32_t *remaining_cnt, return false; } -static void * +static uint32_t mlx5_vdpa_c_thread_handle(void *arg) { struct mlx5_vdpa_conf_thread_mng *multhrd = arg; - pthread_t thread_id = pthread_self(); struct mlx5_vdpa_virtq *virtq; struct mlx5_vdpa_priv *priv; struct mlx5_vdpa_task task; @@ -112,10 +111,10 @@ mlx5_vdpa_c_thread_handle(void *arg) for (thrd_idx = 0; thrd_idx < multhrd->max_thrds; thrd_idx++) - if (multhrd->cthrd[thrd_idx].tid == thread_id) + if (rte_thread_equal(multhrd->cthrd[thrd_idx].tid, rte_thread_self())) break; if (thrd_idx >= multhrd->max_thrds) - return NULL; + return 1; rng = multhrd->cthrd[thrd_idx].rng; while (1) { task_num = mlx5_vdpa_c_thrd_ring_dequeue_bulk(rng, @@ -227,16 +226,17 @@ mlx5_vdpa_c_thread_handle(void *arg) __atomic_fetch_sub(task.remaining_cnt, 1, __ATOMIC_RELAXED); } - return NULL; + return 0; } static void mlx5_vdpa_c_thread_destroy(uint32_t thrd_idx, bool need_unlock) { - if (conf_thread_mng.cthrd[thrd_idx].tid) { - pthread_cancel(conf_thread_mng.cthrd[thrd_idx].tid); - pthread_join(conf_thread_mng.cthrd[thrd_idx].tid, NULL); - conf_thread_mng.cthrd[thrd_idx].tid = 0; + pthread_t *tid = (pthread_t *)&conf_thread_mng.cthrd[thrd_idx].tid.opaque_id; + if (*tid != 0) { + pthread_cancel(*tid); + rte_thread_join(conf_thread_mng.cthrd[thrd_idx].tid, NULL); + *tid = 0; if (need_unlock) pthread_mutex_init(&conf_thread_mng.cthrd_lock, NULL); } @@ -247,30 +247,14 @@ mlx5_vdpa_c_thread_destroy(uint32_t thrd_idx, bool need_unlock) } static int -mlx5_vdpa_c_thread_create(int cpu_core) +mlx5_vdpa_c_thread_create(void) { - const struct sched_param sp = { - .sched_priority = sched_get_priority_max(SCHED_RR), - }; - rte_cpuset_t cpuset; - pthread_attr_t attr; uint32_t thrd_idx; uint32_t ring_num; - char name[32]; + char name[RTE_RING_NAMESIZE]; int ret; pthread_mutex_lock(&conf_thread_mng.cthrd_lock); - pthread_attr_init(&attr); - ret = pthread_attr_setschedpolicy(&attr, SCHED_RR); - if (ret) { - DRV_LOG(ERR, "Failed to set thread sched policy = RR."); - goto c_thread_err; - } - ret = pthread_attr_setschedparam(&attr, &sp); - if (ret) { - DRV_LOG(ERR, "Failed to set thread priority."); - goto c_thread_err; - } ring_num = MLX5_VDPA_MAX_TASKS_PER_THRD / conf_thread_mng.max_thrds; if (!ring_num) { DRV_LOG(ERR, "Invalid ring number for thread."); @@ -291,35 +275,15 @@ mlx5_vdpa_c_thread_create(int cpu_core) thrd_idx); goto c_thread_err; } - ret = pthread_create(&conf_thread_mng.cthrd[thrd_idx].tid, - &attr, mlx5_vdpa_c_thread_handle, - (void *)&conf_thread_mng); + snprintf(name, RTE_THREAD_INTERNAL_NAME_SIZE, "vmlx5-c%d", thrd_idx); + ret = rte_thread_create_internal_control(&conf_thread_mng.cthrd[thrd_idx].tid, + name, + mlx5_vdpa_c_thread_handle, &conf_thread_mng); if (ret) { DRV_LOG(ERR, "Failed to create vdpa multi-threads %d.", thrd_idx); goto c_thread_err; } - CPU_ZERO(&cpuset); - if (cpu_core != -1) - CPU_SET(cpu_core, &cpuset); - else - cpuset = rte_lcore_cpuset(rte_get_main_lcore()); - ret = pthread_setaffinity_np( - conf_thread_mng.cthrd[thrd_idx].tid, - sizeof(cpuset), &cpuset); - if (ret) { - DRV_LOG(ERR, "Failed to set thread affinity for " - "vdpa multi-threads %d.", thrd_idx); - goto c_thread_err; - } - snprintf(name, sizeof(name), "dpdk-vmlx5-c%d", thrd_idx); - ret = pthread_setname_np( - conf_thread_mng.cthrd[thrd_idx].tid, name); - if (ret) - DRV_LOG(ERR, "Failed to set vdpa multi-threads name %s.", - name); - else - DRV_LOG(DEBUG, "Thread name: %s.", name); pthread_cond_init(&conf_thread_mng.cthrd[thrd_idx].c_cond, NULL); } @@ -334,10 +298,10 @@ mlx5_vdpa_c_thread_create(int cpu_core) } int -mlx5_vdpa_mult_threads_create(int cpu_core) +mlx5_vdpa_mult_threads_create(void) { pthread_mutex_init(&conf_thread_mng.cthrd_lock, NULL); - if (mlx5_vdpa_c_thread_create(cpu_core)) { + if (mlx5_vdpa_c_thread_create()) { DRV_LOG(ERR, "Cannot create vDPA configuration threads."); mlx5_vdpa_mult_threads_destroy(false); return -1; diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_event.c b/drivers/vdpa/mlx5/mlx5_vdpa_event.c index c486ded598..fa26471b18 100644 --- a/drivers/vdpa/mlx5/mlx5_vdpa_event.c +++ b/drivers/vdpa/mlx5/mlx5_vdpa_event.c @@ -284,7 +284,7 @@ mlx5_vdpa_event_wait(struct mlx5_vdpa_priv *priv __rte_unused) return NULL; } -static void * +static uint32_t mlx5_vdpa_event_handle(void *arg) { struct mlx5_vdpa_priv *priv = arg; @@ -324,7 +324,7 @@ mlx5_vdpa_event_handle(void *arg) } mlx5_vdpa_timer_sleep(priv, max); } - return NULL; + return 0; case MLX5_VDPA_EVENT_MODE_ONLY_INTERRUPT: do { virtq = mlx5_vdpa_event_wait(priv); @@ -336,9 +336,9 @@ mlx5_vdpa_event_handle(void *arg) pthread_mutex_unlock(&virtq->virtq_lock); } } while (1); - return NULL; + return 0; default: - return NULL; + return 0; } } @@ -503,54 +503,31 @@ int mlx5_vdpa_cqe_event_setup(struct mlx5_vdpa_priv *priv) { int ret; - rte_cpuset_t cpuset; - pthread_attr_t *attrp = NULL; - pthread_attr_t attr; - char name[16]; - const struct sched_param sp = { - .sched_priority = sched_get_priority_max(SCHED_RR) - 1, - }; + rte_thread_attr_t attr; + char name[RTE_THREAD_INTERNAL_NAME_SIZE]; if (!priv->eventc) /* All virtqs are in poll mode. */ return 0; - ret = pthread_attr_init(&attr); + ret = rte_thread_attr_init(&attr); if (ret != 0) { DRV_LOG(ERR, "Failed to initialize thread attributes"); goto out; } - attrp = &attr; - ret = pthread_attr_setschedpolicy(attrp, SCHED_RR); - if (ret) { - DRV_LOG(ERR, "Failed to set thread sched policy = RR."); - goto out; - } - ret = pthread_attr_setschedparam(attrp, &sp); - if (ret) { - DRV_LOG(ERR, "Failed to set thread priority."); - goto out; - } - ret = pthread_create(&priv->timer_tid, attrp, mlx5_vdpa_event_handle, - (void *)priv); - if (ret) { - DRV_LOG(ERR, "Failed to create timer thread."); - goto out; - } - CPU_ZERO(&cpuset); + attr.priority = RTE_THREAD_PRIORITY_REALTIME_CRITICAL; if (priv->event_core != -1) - CPU_SET(priv->event_core, &cpuset); + CPU_SET(priv->event_core, &attr.cpuset); else - cpuset = rte_lcore_cpuset(rte_get_main_lcore()); - ret = pthread_setaffinity_np(priv->timer_tid, sizeof(cpuset), &cpuset); - if (ret) { - DRV_LOG(ERR, "Failed to set thread affinity."); + attr.cpuset = rte_lcore_cpuset(rte_get_main_lcore()); + ret = rte_thread_create(&priv->timer_tid, + &attr, mlx5_vdpa_event_handle, priv); + if (ret != 0) { + DRV_LOG(ERR, "Failed to create timer thread."); goto out; } - snprintf(name, sizeof(name), "dpdk-vmlx5-%d", priv->vid); - rte_thread_set_name((rte_thread_t){(uintptr_t)priv->timer_tid}, name); + snprintf(name, sizeof(name), "vmlx5-%d", priv->vid); + rte_thread_set_prefixed_name(priv->timer_tid, name); out: - if (attrp != NULL) - pthread_attr_destroy(attrp); if (ret != 0) return -1; return 0; @@ -560,19 +537,18 @@ void mlx5_vdpa_cqe_event_unset(struct mlx5_vdpa_priv *priv) { struct mlx5_vdpa_virtq *virtq; - void *status; int i; - if (priv->timer_tid) { - pthread_cancel(priv->timer_tid); - pthread_join(priv->timer_tid, &status); + if (priv->timer_tid.opaque_id != 0) { + pthread_cancel((pthread_t)priv->timer_tid.opaque_id); + rte_thread_join(priv->timer_tid, NULL); /* The mutex may stay locked after event thread cancel, initiate it. */ for (i = 0; i < priv->nr_virtqs; i++) { virtq = &priv->virtqs[i]; pthread_mutex_init(&virtq->virtq_lock, NULL); } } - priv->timer_tid = 0; + priv->timer_tid.opaque_id = 0; } void diff --git a/drivers/vdpa/sfc/sfc_vdpa_ops.c b/drivers/vdpa/sfc/sfc_vdpa_ops.c index f63af7d478..00f9a4b04c 100644 --- a/drivers/vdpa/sfc/sfc_vdpa_ops.c +++ b/drivers/vdpa/sfc/sfc_vdpa_ops.c @@ -567,7 +567,7 @@ sfc_vdpa_get_protocol_features(struct rte_vdpa_device *vdpa_dev, return 0; } -static void * +static uint32_t sfc_vdpa_notify_ctrl(void *arg) { struct sfc_vdpa_ops_data *ops_data; @@ -575,7 +575,7 @@ sfc_vdpa_notify_ctrl(void *arg) ops_data = arg; if (ops_data == NULL) - return NULL; + return 0; sfc_vdpa_adapter_lock(sfc_vdpa_adapter_by_dev_handle(ops_data->dev_handle)); @@ -588,7 +588,7 @@ sfc_vdpa_notify_ctrl(void *arg) sfc_vdpa_adapter_unlock(sfc_vdpa_adapter_by_dev_handle(ops_data->dev_handle)); - return NULL; + return 0; } static int @@ -603,8 +603,8 @@ sfc_vdpa_setup_notify_ctrl(struct sfc_vdpa_ops_data *ops_data) * dead lock scenario when multiple VFs are used in single vdpa * application and multiple VFs are passed to a single VM. */ - ret = pthread_create(&ops_data->notify_tid, NULL, - sfc_vdpa_notify_ctrl, ops_data); + ret = rte_thread_create_internal_control(&ops_data->notify_tid, + "sfc-vdpa", sfc_vdpa_notify_ctrl, ops_data); if (ret != 0) { sfc_vdpa_err(ops_data->dev_handle, "failed to create notify_ctrl thread: %s", @@ -690,15 +690,14 @@ sfc_vdpa_dev_close(int vid) sfc_vdpa_adapter_lock(sfc_vdpa_adapter_by_dev_handle(ops_data->dev_handle)); if (ops_data->is_notify_thread_started == true) { - void *status; - ret = pthread_cancel(ops_data->notify_tid); + ret = pthread_cancel((pthread_t)ops_data->notify_tid.opaque_id); if (ret != 0) { sfc_vdpa_err(ops_data->dev_handle, "failed to cancel notify_ctrl thread: %s", rte_strerror(ret)); } - ret = pthread_join(ops_data->notify_tid, &status); + ret = rte_thread_join(ops_data->notify_tid, NULL); if (ret != 0) { sfc_vdpa_err(ops_data->dev_handle, "failed to join terminated notify_ctrl thread: %s", diff --git a/drivers/vdpa/sfc/sfc_vdpa_ops.h b/drivers/vdpa/sfc/sfc_vdpa_ops.h index 5c8e352de3..10ea239434 100644 --- a/drivers/vdpa/sfc/sfc_vdpa_ops.h +++ b/drivers/vdpa/sfc/sfc_vdpa_ops.h @@ -6,6 +6,7 @@ #define _SFC_VDPA_OPS_H #include +#include #define SFC_VDPA_MAX_QUEUE_PAIRS 8 @@ -48,7 +49,7 @@ struct sfc_vdpa_ops_data { struct rte_vdpa_device *vdpa_dev; enum sfc_vdpa_context vdpa_context; enum sfc_vdpa_state state; - pthread_t notify_tid; + rte_thread_t notify_tid; bool is_notify_thread_started; uint64_t dev_features; -- 2.42.0