From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id C80F01B551 for ; Tue, 8 Jan 2019 17:53:53 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jan 2019 08:53:52 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,454,1539673200"; d="scan'208";a="104964394" Received: from irsmsx151.ger.corp.intel.com ([163.33.192.59]) by orsmga007.jf.intel.com with ESMTP; 08 Jan 2019 08:53:51 -0800 Received: from irsmsx107.ger.corp.intel.com ([169.254.10.127]) by IRSMSX151.ger.corp.intel.com ([169.254.4.172]) with mapi id 14.03.0415.000; Tue, 8 Jan 2019 16:53:50 +0000 From: "Burakov, Anatoly" To: Kevin Traynor CC: "Varghese, Vipin" , dpdk stable Thread-Topic: patch 'eal: clean up unused files on initialization' has been queued to LTS release 18.11.1 Thread-Index: AQHUpDEcWEuiX6YahUO+OuEs9botkqWlnO3w Date: Tue, 8 Jan 2019 16:53:49 +0000 Message-ID: References: <20190104132455.15170-1-ktraynor@redhat.com> <20190104132455.15170-9-ktraynor@redhat.com> In-Reply-To: <20190104132455.15170-9-ktraynor@redhat.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMDc0YzE4NjEtMTIzYi00MWZlLWE0MjQtZDcwOTQ4MjNhYWMwIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiZFdZOUpuNVM3WVl5dHZDRVZzVWtvUzZ0TVhLMkNwditTNDhFSVhSMFJXS2JJK0pJUDFlb0E4TWx6cDdcL1QrT24ifQ== dlp-product: dlpe-windows dlp-version: 11.0.400.15 dlp-reaction: no-action x-originating-ip: [163.33.239.181] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-stable] patch 'eal: clean up unused files on initialization' has been queued to LTS release 18.11.1 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jan 2019 16:53:54 -0000 FYI, There's an issue with this patch on FreeBSD that I'm still investigating. I= 'm hoping to have a fix by tomorrow EOD. Thanks, Anatoly > -----Original Message----- > From: Kevin Traynor [mailto:ktraynor@redhat.com] > Sent: Friday, January 4, 2019 1:24 PM > To: Burakov, Anatoly > Cc: Varghese, Vipin ; dpdk stable > > Subject: patch 'eal: clean up unused files on initialization' has been qu= eued to > LTS release 18.11.1 >=20 > Hi, >=20 > FYI, your patch has been queued to LTS release 18.11.1 >=20 > Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. > It will be pushed if I get no objections before 01/11/19. So please shout= if > anyone has objections. >=20 > Also note that after the patch there's a diff of the upstream commit vs t= he > patch applied to the branch. This will indicate if there was any rebasing > needed to apply to the stable branch. If there were code changes for > rebasing > (ie: not only metadata diffs), please double check that the rebase was > correctly done. >=20 > Thanks. >=20 > Kevin Traynor >=20 > --- > From 8c95205c36c6872e2a96a70bd0044d91cbe1792a Mon Sep 17 00:00:00 > 2001 > From: Anatoly Burakov > Date: Tue, 13 Nov 2018 15:54:44 +0000 > Subject: [PATCH] eal: clean up unused files on initialization >=20 > [ upstream commit 0a529578f162df8b16e4eb7423e55570f3d13c97 ] >=20 > When creating process data structures, EAL will create many files in EAL > runtime directory. Because we allow multiple secondary processes to run, > each secondary process gets their own unique file. With many secondary > processes running and exiting on the system, runtime directory will, over > time, create enormous amounts of sockets, fbarray files and other stuff t= hat > just sits there unused because the process that allocated it has died a l= ong > time ago. This may lead to exhaustion of disk (or RAM) space in the runti= me > directory. >=20 > Fix this by removing every unlocked file at initialization that matches e= ither > socket or fbarray naming convention. We cannot be sure of any other files= , > so we'll leave them alone. Also, remove similar code from mp socket code. >=20 > We do it at the end of init, rather than at the beginning, because second= ary > process will use primary process' data structures even if the primary its= elf has > died, and we don't want to remove those before we lock them. >=20 > Bugzilla ID: 106 >=20 > Reported-by: Vipin Varghese > Signed-off-by: Anatoly Burakov > --- > lib/librte_eal/bsdapp/eal/eal.c | 100 ++++++++++++++++++++++++ > lib/librte_eal/common/eal_common_proc.c | 30 ------- > lib/librte_eal/common/eal_filesystem.h | 3 + > lib/librte_eal/linuxapp/eal/eal.c | 99 +++++++++++++++++++++++ > 4 files changed, 202 insertions(+), 30 deletions(-) >=20 > diff --git a/lib/librte_eal/bsdapp/eal/eal.c b/lib/librte_eal/bsdapp/eal/= eal.c > index b8152a75c..41ddb5a22 100644 > --- a/lib/librte_eal/bsdapp/eal/eal.c > +++ b/lib/librte_eal/bsdapp/eal/eal.c > @@ -4,4 +4,6 @@ > */ >=20 > +#include > +#include > #include > #include > @@ -142,4 +144,90 @@ eal_create_runtime_dir(void) } >=20 > +int > +eal_clean_runtime_dir(void) > +{ > + DIR *dir; > + struct dirent *dirent; > + int dir_fd, fd, lck_result; > + static const char * const filters[] =3D { > + "fbarray_*", > + "mp_socket_*" > + }; > + > + /* open directory */ > + dir =3D opendir(runtime_dir); > + if (!dir) { > + RTE_LOG(ERR, EAL, "Unable to open runtime directory > %s\n", > + runtime_dir); > + goto error; > + } > + dir_fd =3D dirfd(dir); > + > + /* lock the directory before doing anything, to avoid races */ > + if (flock(dir_fd, LOCK_EX) < 0) { > + RTE_LOG(ERR, EAL, "Unable to lock runtime directory %s\n", > + runtime_dir); > + goto error; > + } > + > + dirent =3D readdir(dir); > + if (!dirent) { > + RTE_LOG(ERR, EAL, "Unable to read runtime directory %s\n", > + runtime_dir); > + goto error; > + } > + > + while (dirent !=3D NULL) { > + unsigned int f_idx; > + bool skip =3D true; > + > + /* skip files that don't match the patterns */ > + for (f_idx =3D 0; f_idx < RTE_DIM(filters); f_idx++) { > + const char *filter =3D filters[f_idx]; > + > + if (fnmatch(filter, dirent->d_name, 0) =3D=3D 0) { > + skip =3D false; > + break; > + } > + } > + if (skip) { > + dirent =3D readdir(dir); > + continue; > + } > + > + /* try and lock the file */ > + fd =3D openat(dir_fd, dirent->d_name, O_RDONLY); > + > + /* skip to next file */ > + if (fd =3D=3D -1) { > + dirent =3D readdir(dir); > + continue; > + } > + > + /* non-blocking lock */ > + lck_result =3D flock(fd, LOCK_EX | LOCK_NB); > + > + /* if lock succeeds, remove the file */ > + if (lck_result !=3D -1) > + unlinkat(dir_fd, dirent->d_name, 0); > + close(fd); > + dirent =3D readdir(dir); > + } > + > + /* closedir closes dir_fd and drops the lock */ > + closedir(dir); > + return 0; > + > +error: > + if (dir) > + closedir(dir); > + > + RTE_LOG(ERR, EAL, "Error while clearing runtime dir: %s\n", > + strerror(errno)); > + > + return -1; > +} > + > + > const char * > rte_eal_get_runtime_dir(void) > @@ -808,4 +896,16 @@ rte_eal_init(int argc, char **argv) > } >=20 > + /* > + * Clean up unused files in runtime directory. We do this at the end of > + * init and not at the beginning because we want to clean stuff up > + * whether we are primary or secondary process, but we cannot > remove > + * primary process' files because secondary should be able to run > even > + * if primary process is dead. > + */ > + if (eal_clean_runtime_dir() < 0) { > + rte_eal_init_alert("Cannot clear runtime directory\n"); > + return -1; > + } > + > rte_eal_mcfg_complete(); >=20 > diff --git a/lib/librte_eal/common/eal_common_proc.c > b/lib/librte_eal/common/eal_common_proc.c > index 1c3f09aad..6b876590a 100644 > --- a/lib/librte_eal/common/eal_common_proc.c > +++ b/lib/librte_eal/common/eal_common_proc.c > @@ -543,27 +543,4 @@ open_socket_fd(void) } >=20 > -static int > -unlink_sockets(const char *filter) > -{ > - int dir_fd; > - DIR *mp_dir; > - struct dirent *ent; > - > - mp_dir =3D opendir(mp_dir_path); > - if (!mp_dir) { > - RTE_LOG(ERR, EAL, "Unable to open directory %s\n", > mp_dir_path); > - return -1; > - } > - dir_fd =3D dirfd(mp_dir); > - > - while ((ent =3D readdir(mp_dir))) { > - if (fnmatch(filter, ent->d_name, 0) =3D=3D 0) > - unlinkat(dir_fd, ent->d_name, 0); > - } > - > - closedir(mp_dir); > - return 0; > -} > - > int > rte_mp_channel_init(void) > @@ -604,11 +581,4 @@ rte_mp_channel_init(void) > } >=20 > - if (rte_eal_process_type() =3D=3D RTE_PROC_PRIMARY && > - unlink_sockets(mp_filter)) { > - RTE_LOG(ERR, EAL, "failed to unlink mp sockets\n"); > - close(dir_fd); > - return -1; > - } > - > if (open_socket_fd() < 0) { > close(dir_fd); > diff --git a/lib/librte_eal/common/eal_filesystem.h > b/lib/librte_eal/common/eal_filesystem.h > index 6e0331fdb..64a028db7 100644 > --- a/lib/librte_eal/common/eal_filesystem.h > +++ b/lib/librte_eal/common/eal_filesystem.h > @@ -26,4 +26,7 @@ int > eal_create_runtime_dir(void); >=20 > +int > +eal_clean_runtime_dir(void); > + > #define RUNTIME_CONFIG_FNAME "config" > static inline const char * > diff --git a/lib/librte_eal/linuxapp/eal/eal.c b/lib/librte_eal/linuxapp/= eal/eal.c > index 361744d40..d252c8591 100644 > --- a/lib/librte_eal/linuxapp/eal/eal.c > +++ b/lib/librte_eal/linuxapp/eal/eal.c > @@ -14,5 +14,7 @@ > #include > #include > +#include > #include > +#include > #include > #include > @@ -150,4 +152,89 @@ eal_create_runtime_dir(void) } >=20 > +int > +eal_clean_runtime_dir(void) > +{ > + DIR *dir; > + struct dirent *dirent; > + int dir_fd, fd, lck_result; > + static const char * const filters[] =3D { > + "fbarray_*", > + "mp_socket_*" > + }; > + > + /* open directory */ > + dir =3D opendir(runtime_dir); > + if (!dir) { > + RTE_LOG(ERR, EAL, "Unable to open runtime directory > %s\n", > + runtime_dir); > + goto error; > + } > + dir_fd =3D dirfd(dir); > + > + /* lock the directory before doing anything, to avoid races */ > + if (flock(dir_fd, LOCK_EX) < 0) { > + RTE_LOG(ERR, EAL, "Unable to lock runtime directory %s\n", > + runtime_dir); > + goto error; > + } > + > + dirent =3D readdir(dir); > + if (!dirent) { > + RTE_LOG(ERR, EAL, "Unable to read runtime directory %s\n", > + runtime_dir); > + goto error; > + } > + > + while (dirent !=3D NULL) { > + unsigned int f_idx; > + bool skip =3D true; > + > + /* skip files that don't match the patterns */ > + for (f_idx =3D 0; f_idx < RTE_DIM(filters); f_idx++) { > + const char *filter =3D filters[f_idx]; > + > + if (fnmatch(filter, dirent->d_name, 0) =3D=3D 0) { > + skip =3D false; > + break; > + } > + } > + if (skip) { > + dirent =3D readdir(dir); > + continue; > + } > + > + /* try and lock the file */ > + fd =3D openat(dir_fd, dirent->d_name, O_RDONLY); > + > + /* skip to next file */ > + if (fd =3D=3D -1) { > + dirent =3D readdir(dir); > + continue; > + } > + > + /* non-blocking lock */ > + lck_result =3D flock(fd, LOCK_EX | LOCK_NB); > + > + /* if lock succeeds, remove the file */ > + if (lck_result !=3D -1) > + unlinkat(dir_fd, dirent->d_name, 0); > + close(fd); > + dirent =3D readdir(dir); > + } > + > + /* closedir closes dir_fd and drops the lock */ > + closedir(dir); > + return 0; > + > +error: > + if (dir) > + closedir(dir); > + > + RTE_LOG(ERR, EAL, "Error while clearing runtime dir: %s\n", > + strerror(errno)); > + > + return -1; > +} > + > const char * > rte_eal_get_runtime_dir(void) > @@ -1097,4 +1184,16 @@ rte_eal_init(int argc, char **argv) > } >=20 > + /* > + * Clean up unused files in runtime directory. We do this at the end of > + * init and not at the beginning because we want to clean stuff up > + * whether we are primary or secondary process, but we cannot > remove > + * primary process' files because secondary should be able to run > even > + * if primary process is dead. > + */ > + if (eal_clean_runtime_dir() < 0) { > + rte_eal_init_alert("Cannot clear runtime directory\n"); > + return -1; > + } > + > rte_eal_mcfg_complete(); >=20 > -- > 2.19.0 >=20 > --- > Diff of the applied patch vs upstream commit (please double-check if no= n- > empty: > --- > --- - 2019-01-04 13:23:07.672255523 +0000 > +++ 0009-eal-clean-up-unused-files-on-initialization.patch 2019-01-04 > 13:23:07.000000000 +0000 > @@ -1,8 +1,10 @@ > -From 0a529578f162df8b16e4eb7423e55570f3d13c97 Mon Sep 17 00:00:00 > 2001 > +From 8c95205c36c6872e2a96a70bd0044d91cbe1792a Mon Sep 17 00:00:00 > 2001 > From: Anatoly Burakov > Date: Tue, 13 Nov 2018 15:54:44 +0000 > Subject: [PATCH] eal: clean up unused files on initialization >=20 > +[ upstream commit 0a529578f162df8b16e4eb7423e55570f3d13c97 ] > + > When creating process data structures, EAL will create many files in EA= L > runtime directory. Because we allow multiple secondary processes to run, > each secondary process gets their own unique @@ -24,7 +26,6 @@ before > we lock them. >=20 > Bugzilla ID: 106 > -Cc: stable@dpdk.org >=20 > Reported-by: Vipin Varghese > Signed-off-by: Anatoly Burakov