DPDK patches and discussions
 help / color / mirror / Atom feed
From: David Marchand <david.marchand@redhat.com>
To: Ferruh Yigit <ferruh.yigit@intel.com>
Cc: dev <dev@dpdk.org>, Eelco Chaudron <echaudro@redhat.com>,
	Aaron Conole <aconole@redhat.com>,
	 Maxime Coquelin <maxime.coquelin@redhat.com>,
	"Burakov, Anatoly" <anatoly.burakov@intel.com>,
	 Wenzhuo Lu <wenzhuo.lu@intel.com>,
	Jingjing Wu <jingjing.wu@intel.com>,
	 Bernard Iremonger <bernard.iremonger@intel.com>
Subject: Re: [dpdk-dev] [RFC PATCH] app/testpmd: only lock text pages
Date: Tue, 10 Mar 2020 15:55:36 +0100	[thread overview]
Message-ID: <CAJFAV8zAeWBNH=eP=0-kxMdv+scnpgg4eGq=4Vag1OdsPWWkdw@mail.gmail.com> (raw)
In-Reply-To: <3393023e-4596-5f85-0b3d-b603b6394bf8@intel.com>

On Tue, Mar 10, 2020 at 3:49 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>
> On 3/6/2020 2:48 PM, David Marchand wrote:
> > Since 18.05 and the memory subsystem rework, EAL reserves some big
> > (unused) mappings.
> >
> > In testpmd, we have been locking all pages to avoid page faults during
> > benchmark/performance regression tests [1].
> > However, asking for locking all the pages triggers issues on FreeBSD [2]
> > and becomes really heavy in some Linux configurations (see [3], [4]).
> >
> > This patch changes the behavior so that testpmd only lock pages
> > containing .text by default.
> >
> > 1: https://git.dpdk.org/dpdk/commit/?id=1c036b16c284
> > 2: https://git.dpdk.org/dpdk/commit/?id=fb7b8b32cd95
> > 3: https://bugzilla.redhat.com/show_bug.cgi?id=1786923
> > 4: http://mails.dpdk.org/archives/dev/2020-February/158477.html
> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
>
> <...>
>
> > @@ -3455,6 +3456,42 @@ signal_handler(int signum)
> >       }
> >  }
> >
> > +static void
> > +lock_pages(const void *_addr, size_t _len, const char *prefix)
> > +{
> > +     const void *addr;
> > +     size_t pagesize;
> > +     size_t len;
> > +
> > +     /* While Linux does not care, FreeBSD mlock expects page aligned
> > +      * address (according to the man).
> > +      */
> > +     pagesize = sysconf(_SC_PAGESIZE);
> > +     addr = RTE_PTR_ALIGN_FLOOR(_addr, pagesize);
> > +     len = _len + ((uintptr_t)_addr & (pagesize - 1));
> > +     if (mlock(addr, len)) {
> > +             TESTPMD_LOG(NOTICE, "%s: mlock %p (0x%zx) aligned to %p (0x%zx) failed with error \"%s\"\n",
> > +                     prefix, _addr, _len, addr, len, strerror(errno));
> > +     }
> > +}
> > +
> > +static int
> > +lock_text_cb(struct dl_phdr_info *info, __rte_unused size_t size,
> > +             __rte_unused void *data)
> > +{
> > +     int i;
> > +
> > +     for (i = 0; i < info->dlpi_phnum; i++) {
> > +             void *addr;
> > +
> > +             if (info->dlpi_phdr[i].p_memsz == 0)
> > +                     continue;
> > +             addr = (void *)(info->dlpi_addr + info->dlpi_phdr[i].p_vaddr);
> > +             lock_pages(addr, info->dlpi_phdr[i].p_memsz, info->dlpi_name);
> > +     }
> > +     return 0;
> > +}
>
> +1 to the idea, testpmd initialization was taking too lock without
> '--no-mlockall', but this code looks complex for the application level.
>
> We can do this for testpmd but does all applications need to do something
> similar? If so can we have a solution on eal level instead?

I submitted a patch on the EAL side.
This makes mlockall way lighter, since it skips pages marked with PROT_NONE.
http://patchwork.dpdk.org/patch/66469/


-- 
David Marchand


  reply	other threads:[~2020-03-10 14:55 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-06 14:48 David Marchand
2020-03-10 14:48 ` Ferruh Yigit
2020-03-10 14:55   ` David Marchand [this message]
2020-03-10 15:08     ` Ferruh Yigit
2020-03-10 15:11       ` David Marchand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAJFAV8zAeWBNH=eP=0-kxMdv+scnpgg4eGq=4Vag1OdsPWWkdw@mail.gmail.com' \
    --to=david.marchand@redhat.com \
    --cc=aconole@redhat.com \
    --cc=anatoly.burakov@intel.com \
    --cc=bernard.iremonger@intel.com \
    --cc=dev@dpdk.org \
    --cc=echaudro@redhat.com \
    --cc=ferruh.yigit@intel.com \
    --cc=jingjing.wu@intel.com \
    --cc=maxime.coquelin@redhat.com \
    --cc=wenzhuo.lu@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).