DPDK patches and discussions
 help / color / mirror / Atom feed
From: Anatoly Burakov <anatoly.burakov@intel.com>
To: dev@dpdk.org
Cc: tiwei.bie@intel.com, ray.kinsella@intel.com,
	zhihong.wang@intel.com, maxime.coquelin@redhat.com,
	kuralamudhan.ramakrishnan@intel.com
Subject: [dpdk-dev] [PATCH v3 3/9] mem: raise maximum fd limit unconditionally
Date: Tue,  4 Sep 2018 16:15:44 +0100	[thread overview]
Message-ID: <53a660ffd746c81f59fc3e1f35ac2222a7237641.1536073997.git.anatoly.burakov@intel.com> (raw)
In-Reply-To: <cover.1536073996.git.anatoly.burakov@intel.com>
In-Reply-To: <cover.1536073996.git.anatoly.burakov@intel.com>

Previously, when we allocated hugepages, we closed the fd's corresponding
to them after we've done our mappings. Since we did mmap(), we didn't
actually lose the reference, but file descriptors used for mmap() do not
count against the fd limit. Since we are going to store all of our fd's,
we will hit the fd limit much more often when using smaller page sizes.

Fix this to raise the fd limit to maximum unconditionally.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 lib/librte_eal/linuxapp/eal/eal_memory.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c
index dbf19499e..dfb537f59 100644
--- a/lib/librte_eal/linuxapp/eal/eal_memory.c
+++ b/lib/librte_eal/linuxapp/eal/eal_memory.c
@@ -17,6 +17,7 @@
 #include <sys/stat.h>
 #include <sys/queue.h>
 #include <sys/file.h>
+#include <sys/resource.h>
 #include <unistd.h>
 #include <limits.h>
 #include <sys/ioctl.h>
@@ -2204,6 +2205,25 @@ memseg_secondary_init(void)
 int
 rte_eal_memseg_init(void)
 {
+	/* increase rlimit to maximum */
+	struct rlimit lim;
+
+	if (getrlimit(RLIMIT_NOFILE, &lim) == 0) {
+		/* set limit to maximum */
+		lim.rlim_cur = lim.rlim_max;
+
+		if (setrlimit(RLIMIT_NOFILE, &lim) < 0) {
+			RTE_LOG(DEBUG, EAL, "Setting maximum number of open files failed: %s\n",
+					strerror(errno));
+		} else {
+			RTE_LOG(DEBUG, EAL, "Setting maximum number of open files to %"
+					PRIu64 "\n",
+					(uint64_t)lim.rlim_cur);
+		}
+	} else {
+		RTE_LOG(ERR, EAL, "Cannot get current resource limits\n");
+	}
+
 	return rte_eal_process_type() == RTE_PROC_PRIMARY ?
 #ifndef RTE_ARCH_64
 			memseg_primary_init_32() :
-- 
2.17.1

  parent reply	other threads:[~2018-09-04 15:15 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-08-23 16:59 [dpdk-dev] [PATCH 0/8] Improve running DPDK without hugetlbfs mounpoint Anatoly Burakov
2018-08-23 16:59 ` [dpdk-dev] [PATCH 1/8] fbarray: fix detach in noshconf mode Anatoly Burakov
2018-08-23 16:59 ` [dpdk-dev] [PATCH 2/8] eal: don't allow legacy mode with in-memory mode Anatoly Burakov
2018-08-23 16:59 ` [dpdk-dev] [PATCH 3/8] mem: raise maximum fd limit unconditionally Anatoly Burakov
2018-08-23 16:59 ` [dpdk-dev] [PATCH 4/8] memalloc: rename lock list to fd list Anatoly Burakov
2018-08-23 16:59 ` [dpdk-dev] [PATCH 5/8] memalloc: track page fd's in non-single file mode Anatoly Burakov
2018-08-23 16:59 ` [dpdk-dev] [PATCH 6/8] memalloc: add EAL-internal API to get and set segment fd's Anatoly Burakov
2018-08-23 16:59 ` [dpdk-dev] [PATCH 7/8] mem: add external API to retrieve page fd from EAL Anatoly Burakov
2018-08-23 16:59 ` [dpdk-dev] [PATCH 8/8] mem: support using memfd segments for in-memory mode Anatoly Burakov
2018-08-24  4:39   ` Jerin Jacob
2018-08-24  8:56     ` Burakov, Anatoly
2018-09-04 15:01 ` [dpdk-dev] [PATCH v2 0/9] Improve running DPDK without hugetlbfs mounpoint Anatoly Burakov
2018-09-04 15:15   ` [dpdk-dev] [PATCH v3 " Anatoly Burakov
2018-09-19 13:04     ` Thomas Monjalon
2018-09-19 13:55       ` Burakov, Anatoly
2018-09-19 14:15         ` Thomas Monjalon
2018-09-04 15:15   ` [dpdk-dev] [PATCH v3 1/9] fbarray: fix detach in noshconf mode Anatoly Burakov
2018-09-13 13:00     ` Maxime Coquelin
2018-09-04 15:15   ` [dpdk-dev] [PATCH v3 2/9] eal: don't allow legacy mode with in-memory mode Anatoly Burakov
2018-09-13 13:06     ` Maxime Coquelin
2018-09-17  9:49       ` Burakov, Anatoly
2018-09-04 15:15   ` Anatoly Burakov [this message]
2018-09-13 13:12     ` [dpdk-dev] [PATCH v3 3/9] mem: raise maximum fd limit unconditionally Maxime Coquelin
2018-09-04 15:15   ` [dpdk-dev] [PATCH v3 4/9] memalloc: rename lock list to fd list Anatoly Burakov
2018-09-13 15:19     ` Maxime Coquelin
2018-09-04 15:15   ` [dpdk-dev] [PATCH v3 5/9] memalloc: track page fd's in non-single file mode Anatoly Burakov
2018-09-13 15:56     ` Maxime Coquelin
2018-09-04 15:15   ` [dpdk-dev] [PATCH v3 6/9] memalloc: add EAL-internal API to get and set segment fd's Anatoly Burakov
2018-09-14  7:54     ` Maxime Coquelin
2018-09-17  9:53       ` Burakov, Anatoly
2018-09-04 15:15   ` [dpdk-dev] [PATCH v3 7/9] mem: add external API to retrieve page fd from EAL Anatoly Burakov
2018-09-14  8:00     ` Maxime Coquelin
2018-09-04 15:15   ` [dpdk-dev] [PATCH v3 8/9] mem: allow querying offset into segment fd Anatoly Burakov
2018-09-14  7:57     ` Maxime Coquelin
2018-09-04 15:15   ` [dpdk-dev] [PATCH v3 9/9] mem: support using memfd segments for in-memory mode Anatoly Burakov
2018-09-14  8:06     ` Maxime Coquelin
2018-09-04 15:01 ` [dpdk-dev] [PATCH v2 1/9] fbarray: fix detach in noshconf mode Anatoly Burakov
2018-09-04 15:01 ` [dpdk-dev] [PATCH v2 2/9] eal: don't allow legacy mode with in-memory mode Anatoly Burakov
2018-09-04 15:01 ` [dpdk-dev] [PATCH v2 3/9] mem: raise maximum fd limit unconditionally Anatoly Burakov
2018-09-04 15:01 ` [dpdk-dev] [PATCH v2 4/9] memalloc: rename lock list to fd list Anatoly Burakov
2018-09-04 15:01 ` [dpdk-dev] [PATCH v2 5/9] memalloc: track page fd's in non-single file mode Anatoly Burakov
2018-09-04 15:01 ` [dpdk-dev] [PATCH v2 6/9] memalloc: add EAL-internal API to get and set segment fd's Anatoly Burakov
2018-09-04 15:02 ` [dpdk-dev] [PATCH v2 7/9] mem: add external API to retrieve page fd from EAL Anatoly Burakov
2018-09-04 15:02 ` [dpdk-dev] [PATCH v2 8/9] mem: allow querying offset into segment fd Anatoly Burakov
2018-09-04 15:02 ` [dpdk-dev] [PATCH v2 9/9] mem: support using memfd segments for in-memory mode Anatoly Burakov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=53a660ffd746c81f59fc3e1f35ac2222a7237641.1536073997.git.anatoly.burakov@intel.com \
    --to=anatoly.burakov@intel.com \
    --cc=dev@dpdk.org \
    --cc=kuralamudhan.ramakrishnan@intel.com \
    --cc=maxime.coquelin@redhat.com \
    --cc=ray.kinsella@intel.com \
    --cc=tiwei.bie@intel.com \
    --cc=zhihong.wang@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).