From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 18DC4A04B5; Thu, 10 Sep 2020 18:24:25 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2037D1C113; Thu, 10 Sep 2020 18:24:24 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by dpdk.org (Postfix) with ESMTP id 10A201C113 for ; Thu, 10 Sep 2020 18:24:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1599755062; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=+WLkD0SMvxT2kCaSehxDov4WVf3Fp6lhZfB+ThpCqbE=; b=Q4dLHhoyKANxdJ0srA8KmJ3BPHDw+WjUEjQ6igCMKkm7PZCap/mBom+TY7iAeulB3YHzgp Bzl93noap0hL0slJvdXa38ExCP6gKPQLxk0Cv3GSXQm4Mcy3aEswl/mpQLMH1iB4vwTeYi JDsFbtt8AVXycUV7ysiQGT6LsgAf2Ys= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-245-e6qrVX2nOgK8LPIq0YtQ9Q-1; Thu, 10 Sep 2020 12:24:19 -0400 X-MC-Unique: e6qrVX2nOgK8LPIq0YtQ9Q-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3F9D880EFA5; Thu, 10 Sep 2020 16:24:17 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.205]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5231575122; Thu, 10 Sep 2020 16:24:12 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: maxime.coquelin@redhat.com, sscheink@redhat.com, stable@dpdk.org, Anatoly Burakov Date: Thu, 10 Sep 2020 18:24:07 +0200 Message-Id: <20200910162407.12669-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH] eal/linux: fix memory allocations in containers+SELinux X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This is something we encountered while working in an OpenShift environment with SELinux enabled. In this environment, a DPDK application could create/write to hugepage files but removing them was refused. This resulted in dirty files being reused when starting a new DPDK application and triggered random crashes / erratic behavior. Getting a SELinux setup can be a challenge, and even more if you add containers to the picture :-). So here is a reproducer for the interested testers: # cat >wrap.c < #include #include #include #include #include #include int unlink(const char *pathname) { static int (*orig)(const char *pathname) = NULL; struct stat st; if (orig == NULL) orig = dlsym(RTLD_NEXT, "unlink"); if (strstr(pathname, "rtemap_") != NULL && stat(pathname, &st) == 0) { fprintf(stderr, "### refused unlink for %s\n", pathname); errno = EACCES; return -1; } fprintf(stderr, "### called unlink for %s\n", pathname); return orig(pathname); } int unlinkat(int dirfd, const char *pathname, int flags) { static int (*orig)(int dirfd, const char *pathname, int flags) = NULL; struct stat st; if (orig == NULL) orig = dlsym(RTLD_NEXT, "unlinkat"); if (strstr(pathname, "rtemap_") != NULL && fstatat(dirfd, pathname, &st, flags) == 0) { fprintf(stderr, "### refused unlinkat for %s\n", pathname); errno = EACCES; return -1; } fprintf(stderr, "### called unlinkat for %s\n", pathname); return orig(dirfd, pathname, flags); } EOF # gcc -fPIC -shared -o libwrap.so wrap.c -ldl # \rm /dev/hugepages/rtemap* # # First run is fine # LD_PRELOAD=libwrap.so dpdk-testpmd -w 0000:01:00.0 -- -i [...] Configuring Port 0 (socket 0) Port 0: 24:6E:96:3C:52:D8 Checking link statuses... Done testpmd> # # Second run we have dirty memory # LD_PRELOAD=libwrap.so dpdk-testpmd -w 0000:01:00.0 -- -i [...] ### refused unlinkat for rtemap_0 [...] Port 0 is now not stopped Please stop the ports first Done testpmd> Removing hugepage files is done in multiple places and the memory allocation code is complex. This fix tries to do the minimum and avoids touching other paths. If trying to remove the hugepage file before allocating a page fails, the error is reported to the caller and the user will see a memory allocation error log. Fixes: 582bed1e1d1d ("mem: support mapping hugepages at runtime") Cc: stable@dpdk.org Signed-off-by: David Marchand --- lib/librte_eal/linux/eal_memalloc.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/lib/librte_eal/linux/eal_memalloc.c b/lib/librte_eal/linux/eal_memalloc.c index db60e79975..40a5c4aa1d 100644 --- a/lib/librte_eal/linux/eal_memalloc.c +++ b/lib/librte_eal/linux/eal_memalloc.c @@ -329,6 +329,21 @@ get_seg_fd(char *path, int buflen, struct hugepage_info *hi, fd = fd_list[list_idx].fds[seg_idx]; if (fd < 0) { + /* A primary process is the only one creating these + * files. If there is a leftover that was not cleaned + * by clear_hugedir(), we must *now* make sure to drop + * the file or we will remap old stuff while the rest + * of the code is built on the assumption that a new + * page is clean. + */ + if (rte_eal_process_type() == RTE_PROC_PRIMARY && + unlink(path) == -1 && + errno != ENOENT) { + RTE_LOG(DEBUG, EAL, "%s(): could not remove '%s': %s\n", + __func__, path, strerror(errno)); + return -1; + } + fd = open(path, O_CREAT | O_RDWR, 0600); if (fd < 0) { RTE_LOG(DEBUG, EAL, "%s(): open failed: %s\n", -- 2.23.0