DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH RFC v2 0/2] Reduce DPDK initialization time
@ 2015-11-20  1:53 Zhihong Wang
  2015-11-20  1:53 ` [dpdk-dev] [PATCH RFC v2 1/2] lib/librte_eal: Reduce timer " Zhihong Wang
  2015-11-20  1:53 ` [dpdk-dev] [PATCH RFC v2 2/2] lib/librte_eal: Remove unnecessary hugepage zero-filling Zhihong Wang
  0 siblings, 2 replies; 7+ messages in thread
From: Zhihong Wang @ 2015-11-20  1:53 UTC (permalink / raw)
  To: dev

This RFC patch aims to reduce DPDK initialization time, which is important in cases such as micro service.

Changes are:

1. Reduce timer initialization time

2. Remove unnecessary hugepage zero-filling operations

With this patch:

1. Timer initialization time can be reduced by 4/10 second

2. Memory initialization time can be reduced nearly by half

The 2nd topic has been brought up before in this thread:
http://dpdk.org/dev/patchwork/patch/4219/

--------------
Changes in v2:

1. Use MAP_POPULATE flag to populate page tables

2. Add comments to avoid future misunderstanding

Zhihong Wang (2):
  lib/librte_eal: Reduce timer initialization time
  lib/librte_eal: Remove unnecessary hugepage zero-filling

 lib/librte_eal/linuxapp/eal/eal_memory.c | 20 ++++++--------------
 lib/librte_eal/linuxapp/eal/eal_timer.c  |  2 +-
 2 files changed, 7 insertions(+), 15 deletions(-)

-- 
2.5.0

^ permalink raw reply	[flat|nested] 7+ messages in thread

* [dpdk-dev] [PATCH RFC v2 1/2] lib/librte_eal: Reduce timer initialization time
  2015-11-20  1:53 [dpdk-dev] [PATCH RFC v2 0/2] Reduce DPDK initialization time Zhihong Wang
@ 2015-11-20  1:53 ` Zhihong Wang
  2015-11-20 16:31   ` Stephen Hemminger
  2015-11-20  1:53 ` [dpdk-dev] [PATCH RFC v2 2/2] lib/librte_eal: Remove unnecessary hugepage zero-filling Zhihong Wang
  1 sibling, 1 reply; 7+ messages in thread
From: Zhihong Wang @ 2015-11-20  1:53 UTC (permalink / raw)
  To: dev

Changing from 1/2 second to 1/10 doesn't compromise the precision, and a 4/10 second is worth saving.

Signed-off-by: Zhihong Wang <zhihong.wang@intel.com>
---
 lib/librte_eal/linuxapp/eal/eal_timer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_timer.c b/lib/librte_eal/linuxapp/eal/eal_timer.c
index e0642de..4de0353 100644
--- a/lib/librte_eal/linuxapp/eal/eal_timer.c
+++ b/lib/librte_eal/linuxapp/eal/eal_timer.c
@@ -271,7 +271,7 @@ get_tsc_freq(void)
 #ifdef CLOCK_MONOTONIC_RAW
 #define NS_PER_SEC 1E9
 
-	struct timespec sleeptime = {.tv_nsec = 5E8 }; /* 1/2 second */
+	struct timespec sleeptime = {.tv_nsec = 1E8 }; /* 1/10 second */
 
 	struct timespec t_start, t_end;
 	uint64_t tsc_hz;
-- 
2.5.0

^ permalink raw reply	[flat|nested] 7+ messages in thread

* [dpdk-dev] [PATCH RFC v2 2/2] lib/librte_eal: Remove unnecessary hugepage zero-filling
  2015-11-20  1:53 [dpdk-dev] [PATCH RFC v2 0/2] Reduce DPDK initialization time Zhihong Wang
  2015-11-20  1:53 ` [dpdk-dev] [PATCH RFC v2 1/2] lib/librte_eal: Reduce timer " Zhihong Wang
@ 2015-11-20  1:53 ` Zhihong Wang
  2015-11-20 16:32   ` Stephen Hemminger
  1 sibling, 1 reply; 7+ messages in thread
From: Zhihong Wang @ 2015-11-20  1:53 UTC (permalink / raw)
  To: dev

The kernel fills new allocated (huge) pages with zeros.
DPDK just has to touch the pages to trigger the allocation.

Signed-off-by: Zhihong Wang <zhihong.wang@intel.com>
---
 lib/librte_eal/linuxapp/eal/eal_memory.c | 20 ++++++--------------
 1 file changed, 6 insertions(+), 14 deletions(-)

diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c b/lib/librte_eal/linuxapp/eal/eal_memory.c
index 0de75cd..21a5146 100644
--- a/lib/librte_eal/linuxapp/eal/eal_memory.c
+++ b/lib/librte_eal/linuxapp/eal/eal_memory.c
@@ -399,8 +399,10 @@ map_all_hugepages(struct hugepage_file *hugepg_tbl,
 			return -1;
 		}
 
+		/* map the segment, and populate page tables,
+		 * the kernel fills this segment with zeros */
 		virtaddr = mmap(vma_addr, hugepage_sz, PROT_READ | PROT_WRITE,
-				MAP_SHARED, fd, 0);
+				MAP_SHARED | MAP_POPULATE, fd, 0);
 		if (virtaddr == MAP_FAILED) {
 			RTE_LOG(ERR, EAL, "%s(): mmap failed: %s\n", __func__,
 					strerror(errno));
@@ -410,7 +412,6 @@ map_all_hugepages(struct hugepage_file *hugepg_tbl,
 
 		if (orig) {
 			hugepg_tbl[i].orig_va = virtaddr;
-			memset(virtaddr, 0, hugepage_sz);
 		}
 		else {
 			hugepg_tbl[i].final_va = virtaddr;
@@ -529,22 +530,16 @@ remap_all_hugepages(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi)
 
 			old_addr = vma_addr;
 
-			/* map new, bigger segment */
+			/* map new, bigger segment, and populate page tables,
+			 * the kernel fills this segment with zeros */
 			vma_addr = mmap(vma_addr, total_size,
-					PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
+					PROT_READ | PROT_WRITE, MAP_SHARED | MAP_POPULATE, fd, 0);
 
 			if (vma_addr == MAP_FAILED || vma_addr != old_addr) {
 				RTE_LOG(ERR, EAL, "%s(): mmap failed: %s\n", __func__, strerror(errno));
 				close(fd);
 				return -1;
 			}
-
-			/* touch the page. this is needed because kernel postpones mapping
-			 * creation until the first page fault. with this, we pin down
-			 * the page and it is marked as used and gets into process' pagemap.
-			 */
-			for (offset = 0; offset < total_size; offset += hugepage_sz)
-				*((volatile uint8_t*) RTE_PTR_ADD(vma_addr, offset));
 		}
 
 		/* set shared flock on the file. */
@@ -592,9 +587,6 @@ remap_all_hugepages(struct hugepage_file *hugepg_tbl, struct hugepage_info *hpi)
 			}
 		}
 
-		/* zero out the whole segment */
-		memset(hugepg_tbl[page_idx].final_va, 0, total_size);
-
 		page_idx++;
 	}
 
-- 
2.5.0

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-dev] [PATCH RFC v2 1/2] lib/librte_eal: Reduce timer initialization time
  2015-11-20  1:53 ` [dpdk-dev] [PATCH RFC v2 1/2] lib/librte_eal: Reduce timer " Zhihong Wang
@ 2015-11-20 16:31   ` Stephen Hemminger
  2015-11-23  1:29     ` Wang, Zhihong
  0 siblings, 1 reply; 7+ messages in thread
From: Stephen Hemminger @ 2015-11-20 16:31 UTC (permalink / raw)
  To: Zhihong Wang; +Cc: dev

On Thu, 19 Nov 2015 20:53:47 -0500
Zhihong Wang <zhihong.wang@intel.com> wrote:

> +	struct timespec sleeptime = {.tv_nsec = 1E8 }; /* 1/10 second */

Why not NSEC_PER_SEC / 10

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-dev] [PATCH RFC v2 2/2] lib/librte_eal: Remove unnecessary hugepage zero-filling
  2015-11-20  1:53 ` [dpdk-dev] [PATCH RFC v2 2/2] lib/librte_eal: Remove unnecessary hugepage zero-filling Zhihong Wang
@ 2015-11-20 16:32   ` Stephen Hemminger
  2015-11-23  1:31     ` Wang, Zhihong
  0 siblings, 1 reply; 7+ messages in thread
From: Stephen Hemminger @ 2015-11-20 16:32 UTC (permalink / raw)
  To: Zhihong Wang; +Cc: dev

On Thu, 19 Nov 2015 20:53:48 -0500
Zhihong Wang <zhihong.wang@intel.com> wrote:

> The kernel fills new allocated (huge) pages with zeros.
> DPDK just has to touch the pages to trigger the allocation.
> 
> Signed-off-by: Zhihong Wang <zhihong.wang@intel.com>

The code is correct, and optimized.
Don't you need to update the commit message since you no longer
have to touch the pages.

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-dev] [PATCH RFC v2 1/2] lib/librte_eal: Reduce timer initialization time
  2015-11-20 16:31   ` Stephen Hemminger
@ 2015-11-23  1:29     ` Wang, Zhihong
  0 siblings, 0 replies; 7+ messages in thread
From: Wang, Zhihong @ 2015-11-23  1:29 UTC (permalink / raw)
  To: Stephen Hemminger; +Cc: dev



> -----Original Message-----
> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
> Sent: Saturday, November 21, 2015 12:32 AM
> To: Wang, Zhihong <zhihong.wang@intel.com>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH RFC v2 1/2] lib/librte_eal: Reduce timer
> initialization time
> 
> On Thu, 19 Nov 2015 20:53:47 -0500
> Zhihong Wang <zhihong.wang@intel.com> wrote:
> 
> > +	struct timespec sleeptime = {.tv_nsec = 1E8 }; /* 1/10 second */
> 
> Why not NSEC_PER_SEC / 10

Yes, it looks better using macro :)

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-dev] [PATCH RFC v2 2/2] lib/librte_eal: Remove unnecessary hugepage zero-filling
  2015-11-20 16:32   ` Stephen Hemminger
@ 2015-11-23  1:31     ` Wang, Zhihong
  0 siblings, 0 replies; 7+ messages in thread
From: Wang, Zhihong @ 2015-11-23  1:31 UTC (permalink / raw)
  To: Stephen Hemminger; +Cc: dev



> -----Original Message-----
> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
> Sent: Saturday, November 21, 2015 12:33 AM
> To: Wang, Zhihong <zhihong.wang@intel.com>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH RFC v2 2/2] lib/librte_eal: Remove unnecessary
> hugepage zero-filling
> 
> On Thu, 19 Nov 2015 20:53:48 -0500
> Zhihong Wang <zhihong.wang@intel.com> wrote:
> 
> > The kernel fills new allocated (huge) pages with zeros.
> > DPDK just has to touch the pages to trigger the allocation.
> >
> > Signed-off-by: Zhihong Wang <zhihong.wang@intel.com>
> 
> The code is correct, and optimized.
> Don't you need to update the commit message since you no longer have to touch
> the pages.

Thanks for pointing this out!

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2015-11-23  1:32 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-11-20  1:53 [dpdk-dev] [PATCH RFC v2 0/2] Reduce DPDK initialization time Zhihong Wang
2015-11-20  1:53 ` [dpdk-dev] [PATCH RFC v2 1/2] lib/librte_eal: Reduce timer " Zhihong Wang
2015-11-20 16:31   ` Stephen Hemminger
2015-11-23  1:29     ` Wang, Zhihong
2015-11-20  1:53 ` [dpdk-dev] [PATCH RFC v2 2/2] lib/librte_eal: Remove unnecessary hugepage zero-filling Zhihong Wang
2015-11-20 16:32   ` Stephen Hemminger
2015-11-23  1:31     ` Wang, Zhihong

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).