From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5B2EE45BBB; Thu, 24 Oct 2024 01:19:26 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 27D774028A; Thu, 24 Oct 2024 01:19:26 +0200 (CEST) Received: from mail-lj1-f177.google.com (mail-lj1-f177.google.com [209.85.208.177]) by mails.dpdk.org (Postfix) with ESMTP id 6F6F440267 for ; Thu, 24 Oct 2024 01:19:25 +0200 (CEST) Received: by mail-lj1-f177.google.com with SMTP id 38308e7fff4ca-2fb59652cb9so2728251fa.3 for ; Wed, 23 Oct 2024 16:19:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729725564; x=1730330364; darn=dpdk.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=QMidfDXO3u1WcfQJJBWwNijDDEPo3GczsEXWumuJRqE=; b=mKQou/PWSPhbt5d5LZ3Meq2DDPnVgv7NaQ2+LcFroGRkvUIIOTU0HsC6P9IIs3lofL E48ByhiHDLh6oUht9Ns+DIDdub4WSCsZXTuo2HXIPW94g7kI+i/C9UwB4YzjRFMwwfTl 0pk6qrdz2UnMpZ4ifTv3+4HsDfliMDcKsbXBNNE8eW7MMjnqs5Stsjc5NV0uoV+KZ0GM MyoazCAT5JF0v/rsR7qwC1rn0OVndxjSDoornz5Xpcw+BIVBgSATxHBVmXtSwFF7LazB y4aSd6YMdW2eKL8bZQM+bc5BLhE8Yb4H7KkNOr1Batd4ZVmX9gGwaEsmnqZW/xI00Mpw zgnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729725564; x=1730330364; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=QMidfDXO3u1WcfQJJBWwNijDDEPo3GczsEXWumuJRqE=; b=dkOoM6Txie152nB0xtcGUfq0lgk3u2C7eMmhgHRgHkUiOi3nUjdSqO6CAfoHjZmxte dKXVD7tVDwobGysp1ICIp/a587klgV34w5uepNoF8Sgji9cu3mYCTBAZi9CBPMLnI64Y rGPQa0u3N4zngWJeX0istkWltnUklQPAuz7INV4bAKiifviCnGVaZbufqW7v2QDCZzUe Z8cevU7ufj72XcndvZ/Bsn3oLePsOuhOG+CYxLnPpdOwErC1ujsLQB+/rJa6jMuH0e6L 9eqsg4HOpHKyo/P5K32pbD7lMRK7N2ir4/dP3mEGP4dkIJ45/NpUr9dqDiyPhfsjesZd ASHw== X-Gm-Message-State: AOJu0Yzy5ZHlFDQGFTtV2mO8MRBvJuCh1sBaDl77lETKS7z2dZnwf/I5 Wgxy7+Cw+5Dobe/BIrqWEjL9H+aujw6pLDpA2fKIxWJtL3m7w4RvIrTiv75H X-Google-Smtp-Source: AGHT+IEKghaiznWnsighhOQuzLDZtHS9qfsoNPsHXwq4bSxVAyuez9gK1Wkhg1oabFWiNJ7utLV+Zg== X-Received: by 2002:a05:651c:552:b0:2fa:c59d:1af3 with SMTP id 38308e7fff4ca-2fc9d35a597mr25267111fa.20.1729725563628; Wed, 23 Oct 2024 16:19:23 -0700 (PDT) Received: from sovereign.. (broadband-109-173-43-194.ip.moscow.rt.ru. [109.173.43.194]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-2fb9ad7316asm12163211fa.44.2024.10.23.16.19.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Oct 2024 16:19:21 -0700 (PDT) From: Dmitry Kozlyuk X-Google-Original-From: Dmitry Kozlyuk To: dev@dpdk.org Cc: Dmitry Kozlyuk , Anatoly Burakov , Lewis Donzis Subject: [PATCH] eal: support including mapped memory in core dump Date: Thu, 24 Oct 2024 02:18:59 +0300 Message-Id: <20241023231859.1323727-1-kozlyuk@bifit.com> X-Mailer: git-send-email 2.38.4 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Dmitry Kozlyuk Commit d72e4042c5eb ("mem: exclude unused memory from core dump") unconditionally excluded all hugepage memory managed by DPDK. The rationale was to avoid overly large core dumps generated from reserved (PROT_NONE) but not mapped memory. Mapped hugepages, however, may hold data useful for debugging, even if not being fully used and containing some garbage. Users may want to include those hugepages in core dump. Add `--huge-dump` EAL command-line option to include in core dump all mapped hugepages and also non-hugepage memory allocated with `--no-huge` (as it substitutes for hugepages). Using this option requires more disk storage for core dumps, and also may include sensitive data in core dump, which is why it must be explicitly enabled and a warning is printed. Linux requires /proc/self/coredump_filter adjustment to include hugepages mapped with MAP_SHARED in core dump. Windows EAL requires no change since it automatically excludes reserved memory and includes committed memory. Signed-off-by: Dmitry Kozlyuk --- Lewis, testing on FreeBSD would be appreciated. doc/guides/linux_gsg/eal_args.include.rst | 8 +++ lib/eal/common/eal_common_dynmem.c | 1 + lib/eal/common/eal_common_options.c | 7 ++ lib/eal/common/eal_internal_cfg.h | 2 + lib/eal/common/eal_options.h | 2 + lib/eal/freebsd/eal_memory.c | 19 +++++ lib/eal/linux/eal_memalloc.c | 8 +++ lib/eal/linux/eal_memory.c | 86 +++++++++++++++++++++-- 8 files changed, 129 insertions(+), 4 deletions(-) diff --git a/doc/guides/linux_gsg/eal_args.include.rst b/doc/guides/linux_gsg/eal_args.include.rst index 9cfbf7de84..0cab6b2f16 100644 --- a/doc/guides/linux_gsg/eal_args.include.rst +++ b/doc/guides/linux_gsg/eal_args.include.rst @@ -122,6 +122,14 @@ Memory-related options to system pthread stack size unless the optional size (in kbytes) is specified. +* ``--huge-dump`` + + Include in code dump all hugepages mapped by DPDK either at startup or later, + and ordirary pages mapped by DPDK at startup when ``--no-huge`` is given. + Hugepages increase core dumps size roughly by the amount of memory used. + Dumped memory often contains processed data, which may be sensitive. + Primary and secondary processes can use this option independently. + Debugging options ~~~~~~~~~~~~~~~~~ diff --git a/lib/eal/common/eal_common_dynmem.c b/lib/eal/common/eal_common_dynmem.c index b4dc231940..e6497307b5 100644 --- a/lib/eal/common/eal_common_dynmem.c +++ b/lib/eal/common/eal_common_dynmem.c @@ -7,6 +7,7 @@ #include #include +#include #include #include diff --git a/lib/eal/common/eal_common_options.c b/lib/eal/common/eal_common_options.c index f1a5e329a5..b02a9894ff 100644 --- a/lib/eal/common/eal_common_options.c +++ b/lib/eal/common/eal_common_options.c @@ -105,6 +105,7 @@ eal_long_options[] = { {OPT_NO_TELEMETRY, 0, NULL, OPT_NO_TELEMETRY_NUM }, {OPT_FORCE_MAX_SIMD_BITWIDTH, 1, NULL, OPT_FORCE_MAX_SIMD_BITWIDTH_NUM}, {OPT_HUGE_WORKER_STACK, 2, NULL, OPT_HUGE_WORKER_STACK_NUM }, + {OPT_HUGE_DUMP, 0, NULL, OPT_HUGE_DUMP_NUM }, {0, 0, NULL, 0 } }; @@ -1942,6 +1943,11 @@ eal_parse_common_option(int opt, const char *optarg, return -1; } break; + case OPT_HUGE_DUMP_NUM: + conf->huge_dump = 1; + EAL_LOG(WARNING, "Using --"OPT_HUGE_DUMP + " may result in core dump files which are large or contain sensitive data."); + break; /* don't know what to do, leave this to caller */ default: @@ -2250,6 +2256,7 @@ eal_common_usage(void) " --"OPT_IN_MEMORY" Operate entirely in memory. This will\n" " disable secondary process support\n" " --"OPT_BASE_VIRTADDR" Base virtual address\n" + " --"OPT_HUGE_DUMP" Include hugepages in core dump.\n" " --"OPT_TELEMETRY" Enable telemetry support (on by default)\n" " --"OPT_NO_TELEMETRY" Disable telemetry support\n" " --"OPT_FORCE_MAX_SIMD_BITWIDTH" Force the max SIMD bitwidth\n" diff --git a/lib/eal/common/eal_internal_cfg.h b/lib/eal/common/eal_internal_cfg.h index 167ec501fa..5e03c3952c 100644 --- a/lib/eal/common/eal_internal_cfg.h +++ b/lib/eal/common/eal_internal_cfg.h @@ -103,6 +103,8 @@ struct internal_config { struct simd_bitwidth max_simd_bitwidth; /**< max simd bitwidth path to use */ size_t huge_worker_stack_size; /**< worker thread stack size */ + /** True to include mapped hugepages in coredump. */ + unsigned int huge_dump; }; void eal_reset_internal_config(struct internal_config *internal_cfg); diff --git a/lib/eal/common/eal_options.h b/lib/eal/common/eal_options.h index 3cc9cb6412..604d39417d 100644 --- a/lib/eal/common/eal_options.h +++ b/lib/eal/common/eal_options.h @@ -89,6 +89,8 @@ enum { OPT_FORCE_MAX_SIMD_BITWIDTH_NUM, #define OPT_HUGE_WORKER_STACK "huge-worker-stack" OPT_HUGE_WORKER_STACK_NUM, +#define OPT_HUGE_DUMP "huge-dump" + OPT_HUGE_DUMP_NUM, OPT_LONG_MAX_NUM }; diff --git a/lib/eal/freebsd/eal_memory.c b/lib/eal/freebsd/eal_memory.c index a6f3ba226e..cd99af4b0e 100644 --- a/lib/eal/freebsd/eal_memory.c +++ b/lib/eal/freebsd/eal_memory.c @@ -94,6 +94,12 @@ rte_eal_hugepage_init(void) eal_memseg_list_populate(msl, addr, n_segs); + if (internal_conf->huge_dump) { + if (eal_mem_set_dump(addr, mem_sz, true) < 0) + EAL_LOG(WARNING, "Failed to include pages in core dump (address %p, size %zu): %s", + addr, mem_sz, rte_strerror(rte_errno)); + } + return 0; } @@ -205,6 +211,12 @@ rte_eal_hugepage_init(void) rte_fbarray_set_used(arr, ms_idx); + if (internal_conf->huge_dump) { + if (eal_mem_set_dump(addr, page_sz, true) < 0) + EAL_LOG(WARNING, "Failed to include hugepage in core dump (address %p, size %zu): %s", + addr, page_sz, rte_strerror(rte_errno)); + } + EAL_LOG(INFO, "Mapped memory segment %u @ %p: physaddr:0x%" PRIx64", len %zu", seg_idx++, addr, physaddr, page_sz); @@ -232,6 +244,7 @@ static int attach_segment(const struct rte_memseg_list *msl, const struct rte_memseg *ms, void *arg) { + struct internal_config *internal_conf = eal_get_internal_configuration(); struct attach_walk_args *wa = arg; void *addr; @@ -245,6 +258,12 @@ attach_segment(const struct rte_memseg_list *msl, const struct rte_memseg *ms, return -1; wa->seg_idx++; + if (internal_conf->huge_dump) { + if (eal_mem_set_dump(addr, ms->len, true) < 0) + EAL_LOG(WARNING, "Failed to include hugepage in core dump (address %p, size %zu): %s", + addr, ms->len, rte_strerror(rte_errno)); + } + return 0; } diff --git a/lib/eal/linux/eal_memalloc.c b/lib/eal/linux/eal_memalloc.c index e354efc95d..611b1c65db 100644 --- a/lib/eal/linux/eal_memalloc.c +++ b/lib/eal/linux/eal_memalloc.c @@ -28,6 +28,7 @@ #include /* for hugetlb-related mmap flags */ #include +#include #include #include #include @@ -688,6 +689,13 @@ alloc_seg(struct rte_memseg *ms, void *addr, int socket_id, ms->socket_id = socket_id; ms->flags = dirty ? RTE_MEMSEG_FLAG_DIRTY : 0; + if (internal_conf->huge_dump) { + if (eal_mem_set_dump(addr, alloc_sz, true) < 0) + EAL_LOG(WARNING, + "Failed to include hugepage in coredump (address %p, size %zu): %s", + addr, alloc_sz, rte_strerror(rte_errno)); + } + return 0; mapped: diff --git a/lib/eal/linux/eal_memory.c b/lib/eal/linux/eal_memory.c index 45879ca743..c1cb8d71e7 100644 --- a/lib/eal/linux/eal_memory.c +++ b/lib/eal/linux/eal_memory.c @@ -659,6 +659,7 @@ unmap_unneeded_hugepages(struct hugepage_file *hugepg_tbl, static int remap_segment(struct hugepage_file *hugepages, int seg_start, int seg_end) { + const struct internal_config *internal_conf = eal_get_internal_configuration(); struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; struct rte_memseg_list *msl; struct rte_fbarray *arr; @@ -668,10 +669,9 @@ remap_segment(struct hugepage_file *hugepages, int seg_start, int seg_end) uint64_t page_sz; size_t memseg_len; int socket_id; -#ifndef RTE_ARCH_64 - const struct internal_config *internal_conf = - eal_get_internal_configuration(); -#endif + void *map_addr; + size_t map_len; + page_sz = hugepages[seg_start].size; socket_id = hugepages[seg_start].socket_id; seg_len = seg_end - seg_start; @@ -721,6 +721,9 @@ remap_segment(struct hugepage_file *hugepages, int seg_start, int seg_end) return -1; } + map_addr = RTE_PTR_ADD(msl->base_va, ms_idx * page_sz); + map_len = page_sz * seg_len; + #ifdef RTE_ARCH_PPC_64 /* for PPC64 we go through the list backwards */ for (cur_page = seg_end - 1; cur_page >= seg_start; @@ -793,6 +796,14 @@ remap_segment(struct hugepage_file *hugepages, int seg_start, int seg_end) EAL_LOG(ERR, "Could not store segment fd: %s", rte_strerror(rte_errno)); } + + if (internal_conf->huge_dump) { + if (eal_mem_set_dump(map_addr, map_len, true) < 0) + EAL_LOG(WARNING, + "Failed to include hugepages in core dump (address %p, size %zu): %s", + map_addr, map_len, rte_strerror(rte_errno)); + } + EAL_LOG(DEBUG, "Allocated %" PRIu64 "M on socket %i", (seg_len * page_sz) >> 20, socket_id); return seg_len; @@ -1241,6 +1252,13 @@ eal_legacy_hugepage_init(void) __func__); goto fail; } + + if (internal_conf->huge_dump) { + if (eal_mem_set_dump(prealloc_addr, mem_sz, true) < 0) + EAL_LOG(WARNING, "Failed to include pages in core dump (address %p, size %zu): %s", + prealloc_addr, mem_sz, rte_strerror(rte_errno)); + } + return 0; } @@ -1518,6 +1536,7 @@ getFileSize(int fd) static int eal_legacy_hugepage_attach(void) { + struct internal_config *internal_conf = eal_get_internal_configuration(); struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; struct hugepage_file *hp = NULL; unsigned int num_hp = 0; @@ -1616,6 +1635,12 @@ eal_legacy_hugepage_attach(void) if (eal_memalloc_set_seg_fd(msl_idx, ms_idx, fd) < 0) EAL_LOG(ERR, "Could not store segment fd: %s", rte_strerror(rte_errno)); + + if (internal_conf->huge_dump) { + if (eal_mem_set_dump(map_addr, map_sz, true) < 0) + EAL_LOG(WARNING, "Failed to include hugepage in core dump (address %p, size %zu): %s", + map_addr, map_sz, rte_strerror(rte_errno)); + } } /* unmap the hugepage config file, since we are done using it */ munmap(hp, size); @@ -1650,12 +1675,65 @@ eal_hugepage_attach(void) return 0; } +static int +enable_shared_hugepage_coredump(void) +{ + const char *path = "/proc/self/coredump_filter"; + const unsigned long shared_hugepage_flag = RTE_BIT64(6); + + FILE *f; + uint64_t coredump_filter; + + f = fopen(path, "r"); + if (f == NULL) { + rte_errno = errno; + EAL_LOG(ERR, "Failed to open %s for reading: %s", path, strerror(errno)); + return -1; + } + + if (fscanf(f, "%"SCNx64, &coredump_filter) != 1) { + rte_errno = errno; + EAL_LOG(ERR, "Failed to parse %s: %s", path, strerror(errno)); + fclose(f); + return -1; + } + + fclose(f); + + if (coredump_filter & shared_hugepage_flag) + return 0; + + f = fopen(path, "w"); + if (f == NULL) { + rte_errno = errno; + EAL_LOG(ERR, "Failed to open %s for writing: %s", path, strerror(errno)); + return -1; + } + + coredump_filter |= shared_hugepage_flag; + if (fprintf(f, "%"PRIx64, coredump_filter) <= 0) { + rte_errno = EIO; + EAL_LOG(ERR, "Failed to write %"PRIx64" to %s", coredump_filter, path); + fclose(f); + return -1; + } + + fclose(f); + return 0; +} + int rte_eal_hugepage_init(void) { const struct internal_config *internal_conf = eal_get_internal_configuration(); + if (internal_conf->huge_dump && enable_shared_hugepage_coredump() < 0) { + EAL_LOG(ERR, "Failed to enable shared hugepage core dump: %s", + rte_strerror(rte_errno)); + return -1; + } + return internal_conf->legacy_mem ? eal_legacy_hugepage_init() : eal_dynmem_hugepage_init(); -- 2.38.4