Re: [dpdk-dev] [RFC PATCH] app/testpmd: only lock text pages

[Ferruh Yigit <ferruh.yigit@intel.com>]

On 3/6/2020 2:48 PM, David Marchand wrote:
> Since 18.05 and the memory subsystem rework, EAL reserves some big
> (unused) mappings.
> 
> In testpmd, we have been locking all pages to avoid page faults during
> benchmark/performance regression tests [1].
> However, asking for locking all the pages triggers issues on FreeBSD [2]
> and becomes really heavy in some Linux configurations (see [3], [4]).
> 
> This patch changes the behavior so that testpmd only lock pages
> containing .text by default.
> 
> 1: https://git.dpdk.org/dpdk/commit/?id=1c036b16c284
> 2: https://git.dpdk.org/dpdk/commit/?id=fb7b8b32cd95
> 3: https://bugzilla.redhat.com/show_bug.cgi?id=1786923
> 4: http://mails.dpdk.org/archives/dev/2020-February/158477.html
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>

<...>

> @@ -3455,6 +3456,42 @@ signal_handler(int signum)
>  	}
>  }
>  
> +static void
> +lock_pages(const void *_addr, size_t _len, const char *prefix)
> +{
> +	const void *addr;
> +	size_t pagesize;
> +	size_t len;
> +
> +	/* While Linux does not care, FreeBSD mlock expects page aligned
> +	 * address (according to the man).
> +	 */
> +	pagesize = sysconf(_SC_PAGESIZE);
> +	addr = RTE_PTR_ALIGN_FLOOR(_addr, pagesize);
> +	len = _len + ((uintptr_t)_addr & (pagesize - 1));
> +	if (mlock(addr, len)) {
> +		TESTPMD_LOG(NOTICE, "%s: mlock %p (0x%zx) aligned to %p (0x%zx) failed with error \"%s\"\n",
> +			prefix, _addr, _len, addr, len, strerror(errno));
> +	}
> +}
> +
> +static int
> +lock_text_cb(struct dl_phdr_info *info, __rte_unused size_t size,
> +		__rte_unused void *data)
> +{
> +	int i;
> +
> +	for (i = 0; i < info->dlpi_phnum; i++) {
> +		void *addr;
> +
> +		if (info->dlpi_phdr[i].p_memsz == 0)
> +			continue;
> +		addr = (void *)(info->dlpi_addr + info->dlpi_phdr[i].p_vaddr);
> +		lock_pages(addr, info->dlpi_phdr[i].p_memsz, info->dlpi_name);
> +	}
> +	return 0;
> +}

+1 to the idea, testpmd initialization was taking too lock without
'--no-mlockall', but this code looks complex for the application level.

We can do this for testpmd but does all applications need to do something
similar? If so can we have a solution on eal level instead?