From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7BC81A0544; Tue, 11 Oct 2022 07:23:48 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2402F40F19; Tue, 11 Oct 2022 07:23:48 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by mails.dpdk.org (Postfix) with ESMTP id E871D40146; Tue, 11 Oct 2022 07:23:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1665465826; x=1697001826; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WI5XCgeJKh/COBOTBPza6gpQ4blqvqov51Is6KUkxnw=; b=U2Zy2LfS4B56JHCBw85eC2aRNVtisWEPJhurjmk/BDlBpSzfIv8nRKwb fc+sUbSR/2QSMG5byWPF2CbeRlSumhN+1XvHTYt952ilN4WXaVKeJqpCX hArWaHWfzf2ctML5phwcMQi64YOGAaPyIS2a2hIINjrwE7mzkDzJhq3JU J+UgMV/KKxieXXWsaOJ4G4MzqBClHRIl699/WRv2kneIwnRJGip8Vimm/ HOBuBr56wEkwXDTlWX9QaoLhx8WaTxcTEOI+jShTAS/p4Fct4an1mosLT KlqnuM0+Dg4GOMn8eM8SSwyCkzWGWSoZgLa1VAY63qhYg777orNznpcsC Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10496"; a="390718873" X-IronPort-AV: E=Sophos;i="5.95,175,1661842800"; d="scan'208";a="390718873" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Oct 2022 22:23:43 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10496"; a="955201826" X-IronPort-AV: E=Sophos;i="5.95,175,1661842800"; d="scan'208";a="955201826" Received: from unknown (HELO localhost.localdomain) ([10.239.252.103]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Oct 2022 22:23:42 -0700 From: Zhichao Zeng To: dev@dpdk.org Cc: stable@dpdk.org, yidingx.zhou@intel.com, Zhichao Zeng , Bruce Richardson , Harman Kalra Subject: [PATCH v6] lib/eal: fix segfaults in exiting Date: Tue, 11 Oct 2022 13:25:14 +0800 Message-Id: <20221011052514.1430015-1-zhichaox.zeng@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906025132.160801-1-zhichaox.zeng@intel.com> References: <20220906025132.160801-1-zhichaox.zeng@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The 'eal-intr-thread' is not closed before memory cleanup in the process of exiting. There is a small chance when 'eal-intr-thread' use some pointers, meanwhile the memory was just cleaned, which causes segfaults. This patch closes the 'eal-intr-thread' before memory cleanup in 'rte_eal_cleanup' to avoid segfaults, and adds a flag to avoid executing 'rte_eal_cleanup' in the child process which is forked to execute some test cases(e.g. debug_autotest of dpdk-test). Bugzilla ID: 1006 Cc: stable@dpdk.org Signed-off-by: Zhichao Zeng --- v6: use atomic operation --- v5: simplify patch --- v4: shorten the prompt message and optimize the commit log --- v3: fix rte_eal_cleanup crash in debug_autotest --- v2: add same API for FreeBSD --- lib/eal/common/eal_private.h | 7 +++++++ lib/eal/freebsd/eal.c | 20 ++++++++++++++++++++ lib/eal/freebsd/eal_interrupts.c | 12 ++++++++++++ lib/eal/linux/eal.c | 20 ++++++++++++++++++++ lib/eal/linux/eal_interrupts.c | 12 ++++++++++++ 5 files changed, 71 insertions(+) diff --git a/lib/eal/common/eal_private.h b/lib/eal/common/eal_private.h index 0f4d75bb89..2e3342bd15 100644 --- a/lib/eal/common/eal_private.h +++ b/lib/eal/common/eal_private.h @@ -152,6 +152,13 @@ int rte_eal_tailqs_init(void); */ int rte_eal_intr_init(void); +/** + * Destroy interrupt handling thread. + * + * This function is private to EAL. + */ +void rte_eal_intr_destroy(void); + /** * Close the default log stream * diff --git a/lib/eal/freebsd/eal.c b/lib/eal/freebsd/eal.c index 1b58cd3da6..0839aa211c 100644 --- a/lib/eal/freebsd/eal.c +++ b/lib/eal/freebsd/eal.c @@ -73,6 +73,8 @@ struct lcore_config lcore_config[RTE_MAX_LCORE]; /* used by rte_rdtsc() */ int rte_cycles_vmware_tsc_map; +/* mark process is forked */ +static uint32_t forked_flag; int eal_clean_runtime_dir(void) @@ -575,6 +577,18 @@ static void rte_eal_init_alert(const char *msg) RTE_LOG(ERR, EAL, "%s\n", msg); } +static void +mark_forked(void) +{ + __atomic_add_fetch(&forked_flag, 1, __ATOMIC_RELAXED); +} + +static uint32_t +is_forked(void) +{ + return __atomic_load_n(&forked_flag, __ATOMIC_RELAXED); +} + /* Launch threads, called at application init(). */ int rte_eal_init(int argc, char **argv) @@ -884,16 +898,22 @@ rte_eal_init(int argc, char **argv) eal_mcfg_complete(); + pthread_atfork(NULL, NULL, mark_forked); + return fctret; } int rte_eal_cleanup(void) { + if (is_forked()) + return 0; + struct internal_config *internal_conf = eal_get_internal_configuration(); rte_service_finalize(); rte_mp_channel_cleanup(); + rte_eal_intr_destroy(); eal_bus_cleanup(); rte_trace_save(); eal_trace_fini(); diff --git a/lib/eal/freebsd/eal_interrupts.c b/lib/eal/freebsd/eal_interrupts.c index 9f720bdc8f..cac3859b06 100644 --- a/lib/eal/freebsd/eal_interrupts.c +++ b/lib/eal/freebsd/eal_interrupts.c @@ -648,6 +648,18 @@ rte_eal_intr_init(void) return ret; } +void +rte_eal_intr_destroy(void) +{ + /* cancel the host thread to wait/handle the interrupt */ + pthread_cancel(intr_thread); + pthread_join(intr_thread, NULL); + + /* close kqueue */ + close(kq); + kq = -1; +} + int rte_intr_rx_ctl(struct rte_intr_handle *intr_handle, int epfd, int op, unsigned int vec, void *data) diff --git a/lib/eal/linux/eal.c b/lib/eal/linux/eal.c index e74542fc71..ef15d7e7f0 100644 --- a/lib/eal/linux/eal.c +++ b/lib/eal/linux/eal.c @@ -77,6 +77,8 @@ struct lcore_config lcore_config[RTE_MAX_LCORE]; /* used by rte_rdtsc() */ int rte_cycles_vmware_tsc_map; +/* mark process is forked */ +static uint32_t forked_flag; int eal_clean_runtime_dir(void) @@ -955,6 +957,18 @@ eal_worker_thread_create(unsigned int lcore_id) return ret; } +static void +mark_forked(void) +{ + __atomic_add_fetch(&forked_flag, 1, __ATOMIC_RELAXED); +} + +static uint32_t +is_forked(void) +{ + return __atomic_load_n(&forked_flag, __ATOMIC_RELAXED); +} + /* Launch threads, called at application init(). */ int rte_eal_init(int argc, char **argv) @@ -1325,6 +1339,8 @@ rte_eal_init(int argc, char **argv) eal_mcfg_complete(); + pthread_atfork(NULL, NULL, mark_forked); + return fctret; } @@ -1348,6 +1364,9 @@ mark_freeable(const struct rte_memseg_list *msl, const struct rte_memseg *ms, int rte_eal_cleanup(void) { + if (is_forked()) + return 0; + /* if we're in a primary process, we need to mark hugepages as freeable * so that finalization can release them back to the system. */ @@ -1363,6 +1382,7 @@ rte_eal_cleanup(void) vfio_mp_sync_cleanup(); #endif rte_mp_channel_cleanup(); + rte_eal_intr_destroy(); eal_bus_cleanup(); rte_trace_save(); eal_trace_fini(); diff --git a/lib/eal/linux/eal_interrupts.c b/lib/eal/linux/eal_interrupts.c index d52ec8eb4c..7e9853e8e7 100644 --- a/lib/eal/linux/eal_interrupts.c +++ b/lib/eal/linux/eal_interrupts.c @@ -1199,6 +1199,18 @@ rte_eal_intr_init(void) return ret; } +void +rte_eal_intr_destroy(void) +{ + /* cancel the host thread to wait/handle the interrupt */ + pthread_cancel(intr_thread); + pthread_join(intr_thread, NULL); + + /* close the pipe used by epoll */ + close(intr_pipe.writefd); + close(intr_pipe.readfd); +} + static void eal_intr_proc_rxtx_intr(int fd, const struct rte_intr_handle *intr_handle) { -- 2.25.1