From: Bruce Richardson <bruce.richardson@intel.com>
To: Olivier MATZ <olivier.matz@6wind.com>
Cc: "Wiles, Keith" <keith.wiles@intel.com>,
"Yigit, Ferruh" <ferruh.yigit@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>,
Thomas Monjalon <thomas.monjalon@6wind.com>
Subject: Re: [dpdk-dev] [PATCH] mk: parallelize make config
Date: Tue, 24 Jan 2017 10:02:46 +0000 [thread overview]
Message-ID: <20170124100245.GA151532@bricha3-MOBL3.ger.corp.intel.com> (raw)
In-Reply-To: <20170124094235.7fda08cc@glumotte.dev.6wind.com>
On Tue, Jan 24, 2017 at 09:42:35AM +0100, Olivier MATZ wrote:
> Hi Keith,
>
> On Mon, 23 Jan 2017 17:50:27 +0000, "Wiles, Keith"
> <keith.wiles@intel.com> wrote:
> > > On Jan 23, 2017, at 10:18 AM, Olivier Matz <olivier.matz@6wind.com>
> > > wrote:
> > >
> > > Hi Ferruh,
> > >
> > > On Sun, 22 Jan 2017 01:50:34 +0000, Ferruh Yigit
> > > <ferruh.yigit@intel.com> wrote:
> > >> make config dependency resolving was always running serial,
> > >> parallelize it for better performance.
> > >>
> > >> $ time make T=x86_64-native-linuxapp-gcc config
> > >> real 0m12.633s
> > >>
> > >> $ time make -j8 T=x86_64-native-linuxapp-gcc config
> > >> real 0m1.826s
> > >>
> > >> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> > >
> > > I have a patch that fix the same issue (configuration takes to
> > > long), but done differently. It is more intrusive, since it rework
> > > the way DEPDIRS are used, but it does not require to use -j.
> >
> > I tested the patch and the performance is very quick and seems to
> > work very nicely. I have not looked at the other patch yet are they
> > both compatible and could be combined or not?
>
> No, I don't think they could be combined.
>
> Ferruh's approach looks less risky, since it's a parallelization
> of the config. My approach changes the way dependencies are processed,
> and also how they are declared. I think it's faster, but more intrusive.
>
I think we may hit a case of diminishing returns. On my system, the
config time goes from 12 seconds down to less than 1 second (large
amounts of parallelism). Even with limiting parallelism to just 4
make processes, the time for config drops to 3 seconds.
I don't see huge value in trying to shave off any more time than
that, given the amount of extra complexity involved.
/Bruce
next prev parent reply other threads:[~2017-01-24 10:02 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-01-22 1:50 Ferruh Yigit
2017-01-23 17:18 ` Olivier Matz
2017-01-23 17:19 ` [dpdk-dev] [PATCH] mk: optimize directory dependencies Olivier Matz
2017-01-24 11:19 ` Robin Jarry
2017-01-24 11:26 ` Bruce Richardson
2017-01-24 12:31 ` Robin Jarry
2017-01-24 11:40 ` Jerin Jacob
2017-01-24 12:15 ` Bruce Richardson
2017-01-24 12:56 ` Jerin Jacob
2017-01-24 13:26 ` Richardson, Bruce
2017-01-24 14:50 ` Olivier MATZ
2017-01-24 14:55 ` Wiles, Keith
2017-03-01 11:25 ` Thomas Monjalon
2017-03-01 12:10 ` Bruce Richardson
2017-03-01 12:30 ` Olivier Matz
2017-01-24 13:05 ` Ferruh Yigit
2017-03-17 17:13 ` Olivier Matz
2017-03-17 17:47 ` Robin Jarry
2017-03-20 8:31 ` Olivier Matz
2017-03-24 13:21 ` [dpdk-dev] [PATCH v2] " Olivier Matz
2017-03-27 21:33 ` Thomas Monjalon
2017-03-28 10:34 ` Ferruh Yigit
2017-03-30 8:51 ` Olivier Matz
2017-03-30 9:27 ` Ferruh Yigit
2017-03-30 12:11 ` Olivier Matz
2017-03-30 12:32 ` [dpdk-dev] [PATCH] mk: fix dependencies to optional configs Olivier Matz
2017-03-30 12:37 ` Ferruh Yigit
2017-03-30 13:37 ` Thomas Monjalon
2017-01-23 17:50 ` [dpdk-dev] [PATCH] mk: parallelize make config Wiles, Keith
2017-01-24 8:42 ` Olivier MATZ
2017-01-24 10:02 ` Bruce Richardson [this message]
2017-01-23 19:03 ` Michał Mirosław
2017-01-30 9:41 ` Ferruh Yigit
2017-01-24 10:52 ` Bruce Richardson
2017-01-29 15:29 ` Thomas Monjalon
2017-01-30 9:46 ` Ferruh Yigit
2017-01-30 10:21 ` [dpdk-dev] [PATCH v2] " Ferruh Yigit
2017-01-30 18:13 ` Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170124100245.GA151532@bricha3-MOBL3.ger.corp.intel.com \
--to=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
--cc=keith.wiles@intel.com \
--cc=olivier.matz@6wind.com \
--cc=thomas.monjalon@6wind.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).