DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages
@ 2017-09-12 13:08 Eelco Chaudron
  2017-09-12 14:50 ` Aaron Conole
                   ` (3 more replies)
  0 siblings, 4 replies; 17+ messages in thread
From: Eelco Chaudron @ 2017-09-12 13:08 UTC (permalink / raw)
  To: jingjing.wu; +Cc: dev

Call the mlockall() function, to attempt to lock all of its process
memory into physical RAM, and preventing the kernel from paging any
of its memory to disk.

When using testpmd for performance testing, depending on the code path
taken, we see a couple of page faults in a row. These faults effect
the overall drop-rate of testpmd. On Linux the mlockall() call will
prefault all the pages of testpmd (and the DPDK libraries if linked
dynamically), even without LD_BIND_NOW.

Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
---
 app/test-pmd/testpmd.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 7d4013941..80f3c3e8e 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -38,6 +38,7 @@
 #include <string.h>
 #include <time.h>
 #include <fcntl.h>
+#include <sys/mman.h>
 #include <sys/types.h>
 #include <errno.h>
 
@@ -2292,6 +2293,8 @@ main(int argc, char** argv)
 	signal(SIGINT, signal_handler);
 	signal(SIGTERM, signal_handler);
 
+	mlockall(MCL_CURRENT | MCL_FUTURE);
+
 	diag = rte_eal_init(argc, argv);
 	if (diag < 0)
 		rte_panic("Cannot init EAL\n");
-- 
2.13.5

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages
  2017-09-12 13:08 [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages Eelco Chaudron
@ 2017-09-12 14:50 ` Aaron Conole
  2017-09-12 20:14   ` Thomas Monjalon
  2017-09-13  9:15 ` Maxime Coquelin
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 17+ messages in thread
From: Aaron Conole @ 2017-09-12 14:50 UTC (permalink / raw)
  To: Eelco Chaudron; +Cc: jingjing.wu, dev

Eelco Chaudron <echaudro@redhat.com> writes:

> Call the mlockall() function, to attempt to lock all of its process
> memory into physical RAM, and preventing the kernel from paging any
> of its memory to disk.
>
> When using testpmd for performance testing, depending on the code path
> taken, we see a couple of page faults in a row. These faults effect
> the overall drop-rate of testpmd. On Linux the mlockall() call will
> prefault all the pages of testpmd (and the DPDK libraries if linked
> dynamically), even without LD_BIND_NOW.
>
> Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
> ---

Acked-by: Aaron Conole <aconole@redhat.com>

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages
  2017-09-12 14:50 ` Aaron Conole
@ 2017-09-12 20:14   ` Thomas Monjalon
  2017-09-12 20:29     ` Aaron Conole
  0 siblings, 1 reply; 17+ messages in thread
From: Thomas Monjalon @ 2017-09-12 20:14 UTC (permalink / raw)
  To: Aaron Conole, Eelco Chaudron; +Cc: dev, jingjing.wu, john.mcnamara

12/09/2017 16:50, Aaron Conole:
> Eelco Chaudron <echaudro@redhat.com> writes:
> 
> > Call the mlockall() function, to attempt to lock all of its process
> > memory into physical RAM, and preventing the kernel from paging any
> > of its memory to disk.
> >
> > When using testpmd for performance testing, depending on the code path
> > taken, we see a couple of page faults in a row. These faults effect
> > the overall drop-rate of testpmd. On Linux the mlockall() call will
> > prefault all the pages of testpmd (and the DPDK libraries if linked
> > dynamically), even without LD_BIND_NOW.
> >
> > Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
> 
> Acked-by: Aaron Conole <aconole@redhat.com>

It is interesting, but why make it in testpmd?

Maybe it should be documented in this guide:
	http://dpdk.org/doc/guides/linux_gsg/nic_perf_intel_platform.html

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages
  2017-09-12 20:14   ` Thomas Monjalon
@ 2017-09-12 20:29     ` Aaron Conole
  2017-09-12 22:13       ` Thomas Monjalon
  0 siblings, 1 reply; 17+ messages in thread
From: Aaron Conole @ 2017-09-12 20:29 UTC (permalink / raw)
  To: Thomas Monjalon
  Cc: Aaron Conole, Eelco Chaudron, dev, jingjing.wu, john.mcnamara

Thomas Monjalon <thomas@monjalon.net> writes:

> 12/09/2017 16:50, Aaron Conole:
>> Eelco Chaudron <echaudro@redhat.com> writes:
>> 
>> > Call the mlockall() function, to attempt to lock all of its process
>> > memory into physical RAM, and preventing the kernel from paging any
>> > of its memory to disk.
>> >
>> > When using testpmd for performance testing, depending on the code path
>> > taken, we see a couple of page faults in a row. These faults effect
>> > the overall drop-rate of testpmd. On Linux the mlockall() call will
>> > prefault all the pages of testpmd (and the DPDK libraries if linked
>> > dynamically), even without LD_BIND_NOW.
>> >
>> > Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
>> 
>> Acked-by: Aaron Conole <aconole@redhat.com>
>
> It is interesting, but why make it in testpmd?
>
> Maybe it should be documented in this guide:
> 	http://dpdk.org/doc/guides/linux_gsg/nic_perf_intel_platform.html

Well, I'm not sure what the user would be able to do to get the
prefaulting performance without having a library they use with
LD_PRELOAD and a function with the constructor attribute which does the
same thing, AND export LD_BIND_NOW before linking starts.

The LD_BIND_NOW simply does the symbol resolution, but there's no
guarantee that it will fault all the code pages in to process space, and
without an mlockall(), I'm not sure that there's any kind of guarantee
that they don't get swapped out of resident memory (which also leads to
later page faults).

Maybe I misunderstood the question?

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages
  2017-09-12 20:29     ` Aaron Conole
@ 2017-09-12 22:13       ` Thomas Monjalon
  2017-09-13  8:55         ` Eelco Chaudron
  2017-09-13 12:28         ` Aaron Conole
  0 siblings, 2 replies; 17+ messages in thread
From: Thomas Monjalon @ 2017-09-12 22:13 UTC (permalink / raw)
  To: Aaron Conole; +Cc: Eelco Chaudron, dev, jingjing.wu, john.mcnamara

12/09/2017 22:29, Aaron Conole:
> Thomas Monjalon <thomas@monjalon.net> writes:
> 
> > 12/09/2017 16:50, Aaron Conole:
> >> Eelco Chaudron <echaudro@redhat.com> writes:
> >> 
> >> > Call the mlockall() function, to attempt to lock all of its process
> >> > memory into physical RAM, and preventing the kernel from paging any
> >> > of its memory to disk.
> >> >
> >> > When using testpmd for performance testing, depending on the code path
> >> > taken, we see a couple of page faults in a row. These faults effect
> >> > the overall drop-rate of testpmd. On Linux the mlockall() call will
> >> > prefault all the pages of testpmd (and the DPDK libraries if linked
> >> > dynamically), even without LD_BIND_NOW.
> >> >
> >> > Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
> >> 
> >> Acked-by: Aaron Conole <aconole@redhat.com>
> >
> > It is interesting, but why make it in testpmd?
> >
> > Maybe it should be documented in this guide:
> > 	http://dpdk.org/doc/guides/linux_gsg/nic_perf_intel_platform.html
> 
> Well, I'm not sure what the user would be able to do to get the
> prefaulting performance without having a library they use with
> LD_PRELOAD and a function with the constructor attribute which does the
> same thing, AND export LD_BIND_NOW before linking starts.
> 
> The LD_BIND_NOW simply does the symbol resolution, but there's no
> guarantee that it will fault all the code pages in to process space, and
> without an mlockall(), I'm not sure that there's any kind of guarantee
> that they don't get swapped out of resident memory (which also leads to
> later page faults).
> 
> Maybe I misunderstood the question?

Maybe you misunderstood :)

I was saying that if this improvement applies to applications,
it should be documented in the tuning guide.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages
  2017-09-12 22:13       ` Thomas Monjalon
@ 2017-09-13  8:55         ` Eelco Chaudron
  2017-09-13 12:28         ` Aaron Conole
  1 sibling, 0 replies; 17+ messages in thread
From: Eelco Chaudron @ 2017-09-13  8:55 UTC (permalink / raw)
  To: Thomas Monjalon, Aaron Conole; +Cc: dev, jingjing.wu, john.mcnamara

On 13/09/17 00:13, Thomas Monjalon wrote:
> 12/09/2017 22:29, Aaron Conole:
>> Thomas Monjalon <thomas@monjalon.net> writes:
>>
>>> 12/09/2017 16:50, Aaron Conole:
>>>> Eelco Chaudron <echaudro@redhat.com> writes:
>>>>
>>>>> Call the mlockall() function, to attempt to lock all of its process
>>>>> memory into physical RAM, and preventing the kernel from paging any
>>>>> of its memory to disk.
>>>>>
>>>>> When using testpmd for performance testing, depending on the code path
>>>>> taken, we see a couple of page faults in a row. These faults effect
>>>>> the overall drop-rate of testpmd. On Linux the mlockall() call will
>>>>> prefault all the pages of testpmd (and the DPDK libraries if linked
>>>>> dynamically), even without LD_BIND_NOW.
>>>>>
>>>>> Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
>>>> Acked-by: Aaron Conole <aconole@redhat.com>
>>> It is interesting, but why make it in testpmd?
>>>
>>> Maybe it should be documented in this guide:
>>> 	http://dpdk.org/doc/guides/linux_gsg/nic_perf_intel_platform.html
>> Well, I'm not sure what the user would be able to do to get the
>> prefaulting performance without having a library they use with
>> LD_PRELOAD and a function with the constructor attribute which does the
>> same thing, AND export LD_BIND_NOW before linking starts.
>>
>> The LD_BIND_NOW simply does the symbol resolution, but there's no
>> guarantee that it will fault all the code pages in to process space, and
>> without an mlockall(), I'm not sure that there's any kind of guarantee
>> that they don't get swapped out of resident memory (which also leads to
>> later page faults).
>>
>> Maybe I misunderstood the question?
> Maybe you misunderstood :)
>
> I was saying that if this improvement applies to applications,
> it should be documented in the tuning guide.
>
I'll try to find a good place in the documentation for adding a 
reference to mlockall(), but will send it as a separate documentation patch.

//Eelco

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages
  2017-09-12 13:08 [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages Eelco Chaudron
  2017-09-12 14:50 ` Aaron Conole
@ 2017-09-13  9:15 ` Maxime Coquelin
  2017-09-13  9:39 ` Thomas Monjalon
  2017-09-29  8:07 ` Sergio Gonzalez Monroy
  3 siblings, 0 replies; 17+ messages in thread
From: Maxime Coquelin @ 2017-09-13  9:15 UTC (permalink / raw)
  To: Eelco Chaudron, jingjing.wu; +Cc: dev



On 09/12/2017 03:08 PM, Eelco Chaudron wrote:
> Call the mlockall() function, to attempt to lock all of its process
> memory into physical RAM, and preventing the kernel from paging any
> of its memory to disk.
> 
> When using testpmd for performance testing, depending on the code path
> taken, we see a couple of page faults in a row. These faults effect
> the overall drop-rate of testpmd. On Linux the mlockall() call will
> prefault all the pages of testpmd (and the DPDK libraries if linked
> dynamically), even without LD_BIND_NOW.
> 
> Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
> ---
>   app/test-pmd/testpmd.c | 3 +++
>   1 file changed, 3 insertions(+)
> 

Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>

Thanks,
Maxime

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages
  2017-09-12 13:08 [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages Eelco Chaudron
  2017-09-12 14:50 ` Aaron Conole
  2017-09-13  9:15 ` Maxime Coquelin
@ 2017-09-13  9:39 ` Thomas Monjalon
  2017-09-14  7:22   ` Eelco Chaudron
  2017-09-29  8:07 ` Sergio Gonzalez Monroy
  3 siblings, 1 reply; 17+ messages in thread
From: Thomas Monjalon @ 2017-09-13  9:39 UTC (permalink / raw)
  To: Eelco Chaudron; +Cc: dev, jingjing.wu

12/09/2017 15:08, Eelco Chaudron:
> Call the mlockall() function, to attempt to lock all of its process
> memory into physical RAM, and preventing the kernel from paging any
> of its memory to disk.
> 
> When using testpmd for performance testing, depending on the code path
> taken, we see a couple of page faults in a row. These faults effect
> the overall drop-rate of testpmd. On Linux the mlockall() call will
> prefault all the pages of testpmd (and the DPDK libraries if linked
> dynamically), even without LD_BIND_NOW.

Does it work on FreeBSD?
Is there any drawback?
Do we need to add an option for it?

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages
  2017-09-12 22:13       ` Thomas Monjalon
  2017-09-13  8:55         ` Eelco Chaudron
@ 2017-09-13 12:28         ` Aaron Conole
  1 sibling, 0 replies; 17+ messages in thread
From: Aaron Conole @ 2017-09-13 12:28 UTC (permalink / raw)
  To: Thomas Monjalon; +Cc: Eelco Chaudron, dev, jingjing.wu, john.mcnamara

Thomas Monjalon <thomas@monjalon.net> writes:

> 12/09/2017 22:29, Aaron Conole:
>> Thomas Monjalon <thomas@monjalon.net> writes:
>> 
>> > 12/09/2017 16:50, Aaron Conole:
>> >> Eelco Chaudron <echaudro@redhat.com> writes:
>> >> 
>> >> > Call the mlockall() function, to attempt to lock all of its process
>> >> > memory into physical RAM, and preventing the kernel from paging any
>> >> > of its memory to disk.
>> >> >
>> >> > When using testpmd for performance testing, depending on the code path
>> >> > taken, we see a couple of page faults in a row. These faults effect
>> >> > the overall drop-rate of testpmd. On Linux the mlockall() call will
>> >> > prefault all the pages of testpmd (and the DPDK libraries if linked
>> >> > dynamically), even without LD_BIND_NOW.
>> >> >
>> >> > Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
>> >> 
>> >> Acked-by: Aaron Conole <aconole@redhat.com>
>> >
>> > It is interesting, but why make it in testpmd?
>> >
>> > Maybe it should be documented in this guide:
>> > 	http://dpdk.org/doc/guides/linux_gsg/nic_perf_intel_platform.html
>> 
>> Well, I'm not sure what the user would be able to do to get the
>> prefaulting performance without having a library they use with
>> LD_PRELOAD and a function with the constructor attribute which does the
>> same thing, AND export LD_BIND_NOW before linking starts.
>> 
>> The LD_BIND_NOW simply does the symbol resolution, but there's no
>> guarantee that it will fault all the code pages in to process space, and
>> without an mlockall(), I'm not sure that there's any kind of guarantee
>> that they don't get swapped out of resident memory (which also leads to
>> later page faults).
>> 
>> Maybe I misunderstood the question?
>
> Maybe you misunderstood :)
>
> I was saying that if this improvement applies to applications,
> it should be documented in the tuning guide.

Ahh, okay.  Yep, I agree with all you've written above.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages
  2017-09-13  9:39 ` Thomas Monjalon
@ 2017-09-14  7:22   ` Eelco Chaudron
  2017-09-19  7:28     ` Olivier MATZ
  0 siblings, 1 reply; 17+ messages in thread
From: Eelco Chaudron @ 2017-09-14  7:22 UTC (permalink / raw)
  To: Thomas Monjalon; +Cc: dev, jingjing.wu

On 13/09/17 11:39, Thomas Monjalon wrote:
> 12/09/2017 15:08, Eelco Chaudron:
>> Call the mlockall() function, to attempt to lock all of its process
>> memory into physical RAM, and preventing the kernel from paging any
>> of its memory to disk.
>>
>> When using testpmd for performance testing, depending on the code path
>> taken, we see a couple of page faults in a row. These faults effect
>> the overall drop-rate of testpmd. On Linux the mlockall() call will
>> prefault all the pages of testpmd (and the DPDK libraries if linked
>> dynamically), even without LD_BIND_NOW.
> Does it work on FreeBSD?
I do not have a FreeBSD setup, but from the documentation I've read the 
call is supported by FreeBSD.
If some one has a working setup, please give this patch a quick try.
> Is there any drawback?
> Do we need to add an option for it?
The only drawback I can think of is that with this change memory 
phyiscal memory is consumed as pages are pre-loaded.
For testpmd (just loaded not doing anything) this is 2MB vs 35MB of 
memory used. I do not think this yields an extra option.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages
  2017-09-14  7:22   ` Eelco Chaudron
@ 2017-09-19  7:28     ` Olivier MATZ
  2017-09-21 12:24       ` Eelco Chaudron
  0 siblings, 1 reply; 17+ messages in thread
From: Olivier MATZ @ 2017-09-19  7:28 UTC (permalink / raw)
  To: Eelco Chaudron; +Cc: Thomas Monjalon, dev, jingjing.wu

Hi,

On Thu, Sep 14, 2017 at 09:22:38AM +0200, Eelco Chaudron wrote:
> On 13/09/17 11:39, Thomas Monjalon wrote:
> > 12/09/2017 15:08, Eelco Chaudron:
> > > Call the mlockall() function, to attempt to lock all of its process
> > > memory into physical RAM, and preventing the kernel from paging any
> > > of its memory to disk.
> > > 
> > > When using testpmd for performance testing, depending on the code path
> > > taken, we see a couple of page faults in a row. These faults effect
> > > the overall drop-rate of testpmd. On Linux the mlockall() call will
> > > prefault all the pages of testpmd (and the DPDK libraries if linked
> > > dynamically), even without LD_BIND_NOW.
> > Does it work on FreeBSD?
> I do not have a FreeBSD setup, but from the documentation I've read the call
> is supported by FreeBSD.
> If some one has a working setup, please give this patch a quick try.
> > Is there any drawback?
> > Do we need to add an option for it?
> The only drawback I can think of is that with this change memory phyiscal
> memory is consumed as pages are pre-loaded.
> For testpmd (just loaded not doing anything) this is 2MB vs 35MB of memory
> used. I do not think this yields an extra option.

One small comment: the call to mlockall() will fail if we don't have
the permissions. I guess it's not an issue, but I wonder if we should
log it?

Olivier

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages
  2017-09-19  7:28     ` Olivier MATZ
@ 2017-09-21 12:24       ` Eelco Chaudron
  2017-09-25  7:53         ` Olivier MATZ
  0 siblings, 1 reply; 17+ messages in thread
From: Eelco Chaudron @ 2017-09-21 12:24 UTC (permalink / raw)
  To: Olivier MATZ; +Cc: Thomas Monjalon, dev, jingjing.wu

On 19/09/17 09:28, Olivier MATZ wrote:
> Hi,
>
> On Thu, Sep 14, 2017 at 09:22:38AM +0200, Eelco Chaudron wrote:
>> On 13/09/17 11:39, Thomas Monjalon wrote:
>>> 12/09/2017 15:08, Eelco Chaudron:
>>>> Call the mlockall() function, to attempt to lock all of its process
>>>> memory into physical RAM, and preventing the kernel from paging any
>>>> of its memory to disk.
>>>>
>>>> When using testpmd for performance testing, depending on the code path
>>>> taken, we see a couple of page faults in a row. These faults effect
>>>> the overall drop-rate of testpmd. On Linux the mlockall() call will
>>>> prefault all the pages of testpmd (and the DPDK libraries if linked
>>>> dynamically), even without LD_BIND_NOW.
>>> Does it work on FreeBSD?
>> I do not have a FreeBSD setup, but from the documentation I've read the call
>> is supported by FreeBSD.
>> If some one has a working setup, please give this patch a quick try.
>>> Is there any drawback?
>>> Do we need to add an option for it?
>> The only drawback I can think of is that with this change memory phyiscal
>> memory is consumed as pages are pre-loaded.
>> For testpmd (just loaded not doing anything) this is 2MB vs 35MB of memory
>> used. I do not think this yields an extra option.
> One small comment: the call to mlockall() will fail if we don't have
> the permissions. I guess it's not an issue, but I wonder if we should
> log it?
I did not at add a log, as the rte log subsystem is not yet initialized.
However we could add a printf("WARNING: mlockall() failed with error %s")
like message. What do you think?

> Olivier

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages
  2017-09-21 12:24       ` Eelco Chaudron
@ 2017-09-25  7:53         ` Olivier MATZ
  2017-09-29  7:59           ` Eelco Chaudron
  0 siblings, 1 reply; 17+ messages in thread
From: Olivier MATZ @ 2017-09-25  7:53 UTC (permalink / raw)
  To: Eelco Chaudron; +Cc: Thomas Monjalon, dev, jingjing.wu

On Thu, Sep 21, 2017 at 02:24:28PM +0200, Eelco Chaudron wrote:
> On 19/09/17 09:28, Olivier MATZ wrote:
> > Hi,
> > 
> > On Thu, Sep 14, 2017 at 09:22:38AM +0200, Eelco Chaudron wrote:
> > > On 13/09/17 11:39, Thomas Monjalon wrote:
> > > > 12/09/2017 15:08, Eelco Chaudron:
> > > > > Call the mlockall() function, to attempt to lock all of its process
> > > > > memory into physical RAM, and preventing the kernel from paging any
> > > > > of its memory to disk.
> > > > > 
> > > > > When using testpmd for performance testing, depending on the code path
> > > > > taken, we see a couple of page faults in a row. These faults effect
> > > > > the overall drop-rate of testpmd. On Linux the mlockall() call will
> > > > > prefault all the pages of testpmd (and the DPDK libraries if linked
> > > > > dynamically), even without LD_BIND_NOW.
> > > > Does it work on FreeBSD?
> > > I do not have a FreeBSD setup, but from the documentation I've read the call
> > > is supported by FreeBSD.
> > > If some one has a working setup, please give this patch a quick try.
> > > > Is there any drawback?
> > > > Do we need to add an option for it?
> > > The only drawback I can think of is that with this change memory phyiscal
> > > memory is consumed as pages are pre-loaded.
> > > For testpmd (just loaded not doing anything) this is 2MB vs 35MB of memory
> > > used. I do not think this yields an extra option.
> > One small comment: the call to mlockall() will fail if we don't have
> > the permissions. I guess it's not an issue, but I wonder if we should
> > log it?
> I did not at add a log, as the rte log subsystem is not yet initialized.
> However we could add a printf("WARNING: mlockall() failed with error %s")
> like message. What do you think?

Since it's not critical, maybe NOTICE is better than WARNING.

One more question: what would be the drawback of calling
mlockall() after eal_init()? (rte_log would be initialized)


Thanks,
Olivier

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages
  2017-09-25  7:53         ` Olivier MATZ
@ 2017-09-29  7:59           ` Eelco Chaudron
  0 siblings, 0 replies; 17+ messages in thread
From: Eelco Chaudron @ 2017-09-29  7:59 UTC (permalink / raw)
  To: Olivier MATZ; +Cc: Thomas Monjalon, dev, jingjing.wu

On 25/09/17 09:53, Olivier MATZ wrote:
>> [SNIP]
>> I did not at add a log, as the rte log subsystem is not yet initialized.
>> However we could add a printf("WARNING: mlockall() failed with error %s")
>> like message. What do you think?
> Since it's not critical, maybe NOTICE is better than WARNING.
>
> One more question: what would be the drawback of calling
> mlockall() after eal_init()? (rte_log would be initialized)
>
Calling mlockall() after eal_init() does seem to have the same effect.
I'll send out a v2 patch later today with a rte_log(NOTICE).

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages
  2017-09-12 13:08 [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages Eelco Chaudron
                   ` (2 preceding siblings ...)
  2017-09-13  9:39 ` Thomas Monjalon
@ 2017-09-29  8:07 ` Sergio Gonzalez Monroy
  2017-09-29  8:15   ` Eelco Chaudron
  3 siblings, 1 reply; 17+ messages in thread
From: Sergio Gonzalez Monroy @ 2017-09-29  8:07 UTC (permalink / raw)
  To: Eelco Chaudron, jingjing.wu; +Cc: dev

On 12/09/2017 14:08, Eelco Chaudron wrote:
> Call the mlockall() function, to attempt to lock all of its process
> memory into physical RAM, and preventing the kernel from paging any
> of its memory to disk.
>
> When using testpmd for performance testing, depending on the code path
> taken, we see a couple of page faults in a row. These faults effect
> the overall drop-rate of testpmd. On Linux the mlockall() call will
> prefault all the pages of testpmd (and the DPDK libraries if linked
> dynamically), even without LD_BIND_NOW.
>
> Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
> ---

When used for performance testing using hugepages or --no-huge option?

Thanks,
Sergio

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages
  2017-09-29  8:07 ` Sergio Gonzalez Monroy
@ 2017-09-29  8:15   ` Eelco Chaudron
  2017-09-29  9:27     ` Sergio Gonzalez Monroy
  0 siblings, 1 reply; 17+ messages in thread
From: Eelco Chaudron @ 2017-09-29  8:15 UTC (permalink / raw)
  To: Sergio Gonzalez Monroy, jingjing.wu; +Cc: dev

On 29/09/17 10:07, Sergio Gonzalez Monroy wrote:
> On 12/09/2017 14:08, Eelco Chaudron wrote:
>> Call the mlockall() function, to attempt to lock all of its process
>> memory into physical RAM, and preventing the kernel from paging any
>> of its memory to disk.
>>
>> When using testpmd for performance testing, depending on the code path
>> taken, we see a couple of page faults in a row. These faults effect
>> the overall drop-rate of testpmd. On Linux the mlockall() call will
>> prefault all the pages of testpmd (and the DPDK libraries if linked
>> dynamically), even without LD_BIND_NOW.
>>
>> Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
>> ---
>
> When used for performance testing using hugepages or --no-huge option?
This is independent of huge pages, its for the text (code) sections.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages
  2017-09-29  8:15   ` Eelco Chaudron
@ 2017-09-29  9:27     ` Sergio Gonzalez Monroy
  0 siblings, 0 replies; 17+ messages in thread
From: Sergio Gonzalez Monroy @ 2017-09-29  9:27 UTC (permalink / raw)
  To: echaudro, jingjing.wu; +Cc: dev

On 29/09/2017 09:15, Eelco Chaudron wrote:
> On 29/09/17 10:07, Sergio Gonzalez Monroy wrote:
>> On 12/09/2017 14:08, Eelco Chaudron wrote:
>>> Call the mlockall() function, to attempt to lock all of its process
>>> memory into physical RAM, and preventing the kernel from paging any
>>> of its memory to disk.
>>>
>>> When using testpmd for performance testing, depending on the code path
>>> taken, we see a couple of page faults in a row. These faults effect
>>> the overall drop-rate of testpmd. On Linux the mlockall() call will
>>> prefault all the pages of testpmd (and the DPDK libraries if linked
>>> dynamically), even without LD_BIND_NOW.
>>>
>>> Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
>>> ---
>>
>> When used for performance testing using hugepages or --no-huge option?
> This is independent of huge pages, its for the text (code) sections.

Understood.

Just for curiosity, how much drop-rate was observed?

Thanks,
Sergio

^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2017-09-29  9:27 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-09-12 13:08 [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages Eelco Chaudron
2017-09-12 14:50 ` Aaron Conole
2017-09-12 20:14   ` Thomas Monjalon
2017-09-12 20:29     ` Aaron Conole
2017-09-12 22:13       ` Thomas Monjalon
2017-09-13  8:55         ` Eelco Chaudron
2017-09-13 12:28         ` Aaron Conole
2017-09-13  9:15 ` Maxime Coquelin
2017-09-13  9:39 ` Thomas Monjalon
2017-09-14  7:22   ` Eelco Chaudron
2017-09-19  7:28     ` Olivier MATZ
2017-09-21 12:24       ` Eelco Chaudron
2017-09-25  7:53         ` Olivier MATZ
2017-09-29  7:59           ` Eelco Chaudron
2017-09-29  8:07 ` Sergio Gonzalez Monroy
2017-09-29  8:15   ` Eelco Chaudron
2017-09-29  9:27     ` Sergio Gonzalez Monroy

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).