From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1310AA056A; Fri, 6 Mar 2020 15:49:16 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 597492BA8; Fri, 6 Mar 2020 15:49:15 +0100 (CET) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by dpdk.org (Postfix) with ESMTP id 9F61FFEB for ; Fri, 6 Mar 2020 15:49:13 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1583506152; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=kWNYYAiRd/5K3urRlYO8EyqNmY07OaiaFQhKTLAfUc8=; b=P10dRiZ5/RYrk5cfDFBnEDrMwzGU5nP0qI5yjt+lH1jZoRi3mmVGtQOOQKpOZV58KEMJFC cgqjk0+Yz2nxDemSXuwOcmtOZr6B0b4hjtFOjbZGbL2YCHj4Gq/9idOyEnGQHhCXiYMecE My8dfJdfwP9PZHrAoUQ/xcD7NqNPvSU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-381--TnG40rDMNamL7KHQmNg_Q-1; Fri, 06 Mar 2020 09:49:09 -0500 X-MC-Unique: -TnG40rDMNamL7KHQmNg_Q-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E40CC73476; Fri, 6 Mar 2020 14:49:07 +0000 (UTC) Received: from dmarchan.remote.csb (ovpn-204-36.brq.redhat.com [10.40.204.36]) by smtp.corp.redhat.com (Postfix) with ESMTP id B5F4F60BEC; Fri, 6 Mar 2020 14:49:04 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: echaudro@redhat.com, aconole@redhat.com, maxime.coquelin@redhat.com, anatoly.burakov@intel.com, Wenzhuo Lu , Jingjing Wu , Bernard Iremonger Date: Fri, 6 Mar 2020 15:48:44 +0100 Message-Id: <20200306144844.27255-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Subject: [dpdk-dev] [RFC PATCH] app/testpmd: only lock text pages X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Since 18.05 and the memory subsystem rework, EAL reserves some big (unused) mappings. In testpmd, we have been locking all pages to avoid page faults during benchmark/performance regression tests [1]. However, asking for locking all the pages triggers issues on FreeBSD [2] and becomes really heavy in some Linux configurations (see [3], [4]). This patch changes the behavior so that testpmd only lock pages containing .text by default. 1: https://git.dpdk.org/dpdk/commit/?id=3D1c036b16c284 2: https://git.dpdk.org/dpdk/commit/?id=3Dfb7b8b32cd95 3: https://bugzilla.redhat.com/show_bug.cgi?id=3D1786923 4: http://mails.dpdk.org/archives/dev/2020-February/158477.html Signed-off-by: David Marchand --- app/test-pmd/parameters.c | 4 +-- app/test-pmd/testpmd.c | 53 +++++++++++++++++++++++++++++++-------- app/test-pmd/testpmd.h | 5 +++- 3 files changed, 49 insertions(+), 13 deletions(-) diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c index 404dba2b2..952371e6c 100644 --- a/app/test-pmd/parameters.c +++ b/app/test-pmd/parameters.c @@ -1304,9 +1304,9 @@ launch_args_parse(int argc, char** argv) =09=09=09if (!strcmp(lgopts[opt_idx].name, "hot-plug")) =09=09=09=09hot_plug =3D 1; =09=09=09if (!strcmp(lgopts[opt_idx].name, "mlockall")) -=09=09=09=09do_mlockall =3D 1; +=09=09=09=09do_mlock =3D TESTPMD_MLOCK_ALL; =09=09=09if (!strcmp(lgopts[opt_idx].name, "no-mlockall")) -=09=09=09=09do_mlockall =3D 0; +=09=09=09=09do_mlock =3D TESTPMD_MLOCK_NONE; =09=09=09if (!strcmp(lgopts[opt_idx].name, =09=09=09=09 "noisy-tx-sw-buffer-size")) { =09=09=09=09n =3D atoi(optarg); diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 035836adf..9a1716321 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -13,6 +13,7 @@ #include #include #include +#include =20 #include #include @@ -390,9 +391,9 @@ uint32_t event_print_mask =3D (UINT32_C(1) << RTE_ETH_E= VENT_UNKNOWN) | =09=09=09 (UINT32_C(1) << RTE_ETH_EVENT_MACSEC) | =09=09=09 (UINT32_C(1) << RTE_ETH_EVENT_INTR_RMV); /* - * Decide if all memory are locked for performance. + * Decide which part of memory is locked for performance. */ -int do_mlockall =3D 0; +int do_mlock =3D TESTPMD_MLOCK_TEXT; =20 /* * NIC bypass mode configuration options. @@ -3455,6 +3456,42 @@ signal_handler(int signum) =09} } =20 +static void +lock_pages(const void *_addr, size_t _len, const char *prefix) +{ +=09const void *addr; +=09size_t pagesize; +=09size_t len; + +=09/* While Linux does not care, FreeBSD mlock expects page aligned +=09 * address (according to the man). +=09 */ +=09pagesize =3D sysconf(_SC_PAGESIZE); +=09addr =3D RTE_PTR_ALIGN_FLOOR(_addr, pagesize); +=09len =3D _len + ((uintptr_t)_addr & (pagesize - 1)); +=09if (mlock(addr, len)) { +=09=09TESTPMD_LOG(NOTICE, "%s: mlock %p (0x%zx) aligned to %p (0x%zx) fail= ed with error \"%s\"\n", +=09=09=09prefix, _addr, _len, addr, len, strerror(errno)); +=09} +} + +static int +lock_text_cb(struct dl_phdr_info *info, __rte_unused size_t size, +=09=09__rte_unused void *data) +{ +=09int i; + +=09for (i =3D 0; i < info->dlpi_phnum; i++) { +=09=09void *addr; + +=09=09if (info->dlpi_phdr[i].p_memsz =3D=3D 0) +=09=09=09continue; +=09=09addr =3D (void *)(info->dlpi_addr + info->dlpi_phdr[i].p_vaddr); +=09=09lock_pages(addr, info->dlpi_phdr[i].p_memsz, info->dlpi_name); +=09} +=09return 0; +} + int main(int argc, char** argv) { @@ -3514,19 +3551,15 @@ main(int argc, char** argv) =09latencystats_enabled =3D 0; #endif =20 -=09/* on FreeBSD, mlockall() is disabled by default */ -#ifdef RTE_EXEC_ENV_FREEBSD -=09do_mlockall =3D 0; -#else -=09do_mlockall =3D 1; -#endif - =09argc -=3D diag; =09argv +=3D diag; =09if (argc > 1) =09=09launch_args_parse(argc, argv); =20 -=09if (do_mlockall && mlockall(MCL_CURRENT | MCL_FUTURE)) { +=09if (do_mlock =3D=3D TESTPMD_MLOCK_TEXT ) { +=09=09dl_iterate_phdr(lock_text_cb, NULL); +=09} else if (do_mlock =3D=3D TESTPMD_MLOCK_ALL && +=09=09=09mlockall(MCL_CURRENT | MCL_FUTURE)) { =09=09TESTPMD_LOG(NOTICE, "mlockall() failed with error \"%s\"\n", =09=09=09strerror(errno)); =09} diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 7a7c73f79..a38e5a1f5 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -343,7 +343,10 @@ extern uint32_t event_print_mask; /**< set by "--print-event xxxx" and "--mask-event xxxx parameters */ extern bool setup_on_probe_event; /**< disabled by port setup-on iterator = */ extern uint8_t hot_plug; /**< enable by "--hot-plug" parameter */ -extern int do_mlockall; /**< set by "--mlockall" or "--no-mlockall" parame= ter */ +#define TESTPMD_MLOCK_NONE 0 +#define TESTPMD_MLOCK_TEXT 1 +#define TESTPMD_MLOCK_ALL 2 +extern int do_mlock; /**< set by "--mlockall" or "--no-mlockall" parameter= */ extern uint8_t clear_ptypes; /**< disabled by set ptype cmd */ =20 #ifdef RTE_LIBRTE_IXGBE_BYPASS --=20 2.23.0