From: Luca Boccassi <luca.boccassi@gmail.com>
To: Ferruh Yigit <ferruh.yigit@intel.com>,
Yipeng Wang <yipeng1.wang@intel.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH 1/7] member: implement main API
Date: Sun, 03 Sep 2017 00:49:34 +0100 [thread overview]
Message-ID: <1504396174.21816.0.camel@gmail.com> (raw)
In-Reply-To: <1504356923.26853.4.camel@gmail.com>
On Sat, 2017-09-02 at 13:55 +0100, Luca Boccassi wrote:
> On Thu, 2017-08-24 at 11:32 +0100, Ferruh Yigit wrote:
> > On 8/24/2017 10:55 AM, Luca Boccassi wrote:
> > > On Thu, 2017-08-24 at 10:35 +0100, Ferruh Yigit wrote:
> > > > On 8/22/2017 11:02 AM, Luca Boccassi wrote:
> > > > > On Mon, 2017-08-21 at 17:19 -0700, Yipeng Wang wrote:
> > > > > > Membership library is an extension and generalization of a
> > > > > > traditional
> > > > > > filter (for example Bloom Filter) structure. In general,
> > > > > > the
> > > > > > Membership
> > > > > > library is a data structure that provides a "set-summary"
> > > > > > and
> > > > > > responds
> > > > > > to set-membership queries of whether a certain element
> > > > > > belongs to
> > > > > > a
> > > > > > set(s). A membership test for an element will return the
> > > > > > set
> > > > > > this
> > > > > > element
> > > > > > belongs to or not-found if the element is never inserted
> > > > > > into
> > > > > > the
> > > > > > set-summary.
> > > > > >
> > > > > > The results of the membership test is not 100% accurate.
> > > > > > Certain
> > > > > > false positive or false negative probability could exist.
> > > > > > However,
> > > > > > comparing to a "full-blown" complete list of elements, a
> > > > > > "set-
> > > > > > summary"
> > > > > > is memory efficient and fast on lookup.
> > > > > >
> > > > > > This patch add the main API definition.
> > > > > >
> > > > > > Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
> > > > > > ---
> > > > > > lib/Makefile | 2 +
> > > > > > lib/librte_eal/common/eal_common_log.c | 1 +
> > > > > > lib/librte_eal/common/include/rte_log.h | 1 +
> > > > > > lib/librte_member/Makefile | 48 +++
> > > > > > lib/librte_member/rte_member.c | 357
> > > > > > +++++++++++++++++++++
> > > > > > lib/librte_member/rte_member.h | 518
> > > > > > +++++++++++++++++++++++++++++++
> > > > > > lib/librte_member/rte_member_version.map | 15 +
> > > > > > 7 files changed, 942 insertions(+)
> > > > > > create mode 100644 lib/librte_member/Makefile
> > > > > > create mode 100644 lib/librte_member/rte_member.c
> > > > > > create mode 100644 lib/librte_member/rte_member.h
> > > > > > create mode 100644
> > > > > > lib/librte_member/rte_member_version.map
> > > > > >
> > > > > > diff --git a/lib/librte_member/Makefile
> > > > > > b/lib/librte_member/Makefile
> > > > > > new file mode 100644
> > > > > > index 0000000..997c825
> > > > > > --- /dev/null
> > > > > > +++ b/lib/librte_member/Makefile
> > > > > > @@ -0,0 +1,48 @@
> > > > > > +# BSD LICENSE
> > > > > > +#
> > > > > > +# Copyright(c) 2017 Intel Corporation. All rights
> > > > > > reserved.
> > > > > > +# All rights reserved.
> > > > > > +#
> > > > > > +# Redistribution and use in source and binary forms,
> > > > > > with
> > > > > > or
> > > > > > without
> > > > > > +# modification, are permitted provided that the
> > > > > > following
> > > > > > conditions
> > > > > > +# are met:
> > > > > > +#
> > > > > > +# * Redistributions of source code must retain the
> > > > > > above
> > > > > > copyright
> > > > > > +# notice, this list of conditions and the following
> > > > > > disclaimer.
> > > > > > +# * Redistributions in binary form must reproduce the
> > > > > > above
> > > > > > copyright
> > > > > > +# notice, this list of conditions and the following
> > > > > > disclaimer
> > > > > > in
> > > > > > +# the documentation and/or other materials provided
> > > > > > with
> > > > > > the
> > > > > > +# distribution.
> > > > > > +# * Neither the name of Intel Corporation nor the
> > > > > > names
> > > > > > of
> > > > > > its
> > > > > > +# contributors may be used to endorse or promote
> > > > > > products
> > > > > > derived
> > > > > > +# from this software without specific prior written
> > > > > > permission.
> > > > > > +#
> > > > > > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
> > > > > > CONTRIBUTORS
> > > > > > +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
> > > > > > INCLUDING,
> > > > > > BUT
> > > > > > NOT
> > > > > > +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
> > > > > > AND
> > > > > > FITNESS FOR
> > > > > > +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
> > > > > > THE
> > > > > > COPYRIGHT
> > > > > > +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
> > > > > > INDIRECT,
> > > > > > INCIDENTAL,
> > > > > > +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
> > > > > > (INCLUDING,
> > > > > > BUT
> > > > > > NOT
> > > > > > +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
> > > > > > SERVICES;
> > > > > > LOSS OF
> > > > > > USE,
> > > > > > +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
> > > > > > CAUSED
> > > > > > AND
> > > > > > ON ANY
> > > > > > +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
> > > > > > LIABILITY,
> > > > > > OR
> > > > > > TORT
> > > > > > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
> > > > > > OUT
> > > > > > OF
> > > > > > THE USE
> > > > > > +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY
> > > > > > OF
> > > > > > SUCH
> > > > > > DAMAGE.
> > > > > > +
> > > > > > +include $(RTE_SDK)/mk/rte.vars.mk
> > > > > > +
> > > > > > +# library name
> > > > > > +LIB = librte_member.a
> > > > > > +
> > > > > > +CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
> > > > > > +
> > > > >
> > > > > This breaks reproducibility as the output directory will be
> > > > > included
> > > > > before the source directory, causing a race - please do
> > > > > something
> > > > > like:
> > > > >
> > > > > CFLAGS := -I$(SRCDIR) $(CFLAGS)
> > > > > CFLAGS += $(WERROR_FLAGS) -O3
> > > >
> > > > Can we remove "-I$(SRCDIR)" completely by first installing
> > > > headers
> > > > and
> > > > later compiling objects, all using $(RTE_OUT) only?
> > > >
> > > > Do you think can this work?
> > >
> > > I'm not sure, it might - but given Bruce's effort to port to
> > > Meson
> > > I'm
> > > not sure it's worth spending a lot of time doing big refactoring
> > > of
> > > the
> > > existing build system
> >
> > Not big refactoring, following seems worked for me, if you would
> > like
> > to
> > test:
> >
> > diff --git a/mk/rte.lib.mk b/mk/rte.lib.mk
> > index 13115d146..643da47da 100644
> > --- a/mk/rte.lib.mk
> > +++ b/mk/rte.lib.mk
> > @@ -59,14 +59,19 @@ endif
> >
> >
> > _BUILD = $(LIB)
> > -_INSTALL = $(INSTALL-FILES-y) $(SYMLINK-FILES-y)
> > $(RTE_OUTPUT)/lib/$(LIB)
> > +PREINSTALL = $(SYMLINK-FILES-y)
> > +_INSTALL = $(INSTALL-FILES-y) $(RTE_OUTPUT)/lib/$(LIB)
> > _CLEAN = doclean
> >
> > .PHONY: all
> > all: install
> >
> > .PHONY: install
> > +ifeq ($(SYMLINK-FILES-y),)
> > install: build _postinstall
> > +else
> > +install: _preinstall build _postinstall
> > +endif
> >
> > _postinstall: build
>
> Easier than I would have thought :-)
>
> I'll give that patch a spin through the repro build grinder next
> week,
> thanks.
Thanks Ferruh, this works in making the build reproducible with the
current CI as an alternative with a fixed build path.
But on the other hand, it will cause the compiler to always embed the
custom build directory in the path to the header in the DWARF .debug
files, rather than the "repository" path:
readelf --debug-dump=line /tmp/69c02db5050024b530a14555b49528c419419f.debug
<cut>
The Directory Table (offset 0x1b):
1 dpdk/lib/librte_eal/linuxapp/eal
2 /usr/include/x86_64-linux-gnu/bits
3 dpdk/lib/librte_eal/common
4 dpdk/debian/build/shared-root/include
5 dpdk/debian/build/shared-root/include/generic
6 /usr/lib/gcc/x86_64-linux-gnu/7/include
7 /usr/include/x86_64-linux-gnu/sys
8 /usr/include
The File Name Table (offset 0x15a):
Entry Dir Time Size Name
1 1 0 0 eal.c
2 2 0 0 stdio2.h
3 3 0 0 eal_filesystem.h
4 2 0 0 fcntl2.h
5 4 0 0 rte_atomic.h
6 5 0 0 rte_atomic.h
7 2 0 0 string3.h
8 4 0 0 rte_cycles.h
9 4 0 0 rte_random.h
10 6 0 0 xmmintrin.h
11 4 0 0 rte_eal_memconfig.h
12 4 0 0 rte_lcore.h
<cut>
note 4 and 5 items in the directory tables, they point to debian/build-
shared-root which is the output directory we use in the package build.
This means that 2 builds that are otherwise exactly the same (same
machine type, same build dependencies) will not be binary identical due
to that difference.
On one hand it's a much smaller and nicer patch and there's the Meson
work which will make both obsolete, on the other hand it's not very
reproducibl-y. What do you think?
--
Kind regards,
Luca Boccassi
next prev parent reply other threads:[~2017-09-02 23:49 UTC|newest]
Thread overview: 81+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-08-22 0:19 [dpdk-dev] [PATCH 0/7] Add Membership Library Yipeng Wang
2017-08-22 0:19 ` [dpdk-dev] [PATCH 1/7] member: implement main API Yipeng Wang
2017-08-22 3:59 ` Stephen Hemminger
2017-08-22 10:02 ` Luca Boccassi
2017-08-24 9:35 ` Ferruh Yigit
2017-08-24 9:55 ` Luca Boccassi
2017-08-24 10:32 ` Ferruh Yigit
2017-09-02 12:55 ` Luca Boccassi
2017-09-02 23:49 ` Luca Boccassi [this message]
2017-08-24 18:38 ` Wang, Yipeng1
2017-09-02 12:54 ` Luca Boccassi
2017-08-22 0:19 ` [dpdk-dev] [PATCH 2/7] member: implement HT mode Yipeng Wang
2017-08-22 0:19 ` [dpdk-dev] [PATCH 3/7] member: implement vBF mode Yipeng Wang
2017-08-22 0:19 ` [dpdk-dev] [PATCH 4/7] member: add AVX for HT mode Yipeng Wang
2017-08-22 0:19 ` [dpdk-dev] [PATCH 5/7] member: enable the library Yipeng Wang
2017-08-22 0:19 ` [dpdk-dev] [PATCH 6/7] test/member: add functional and perf tests Yipeng Wang
2017-08-22 0:19 ` [dpdk-dev] [PATCH 7/7] doc: add membership documentation Yipeng Wang
2017-08-22 4:01 ` [dpdk-dev] [PATCH 0/7] Add Membership Library Stephen Hemminger
2017-08-23 2:58 ` Wang, Yipeng1
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 " Yipeng Wang
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 1/7] member: implement main API Yipeng Wang
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 2/7] member: implement HT mode Yipeng Wang
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 3/7] member: implement vBF mode Yipeng Wang
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 4/7] member: add AVX for HT mode Yipeng Wang
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 5/7] member: enable the library Yipeng Wang
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 6/7] test/member: add functional and perf tests Yipeng Wang
2017-09-02 1:24 ` [dpdk-dev] [PATCH v2 7/7] doc: add membership documentation Yipeng Wang
2017-09-04 13:19 ` Mcnamara, John
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 0/7] Add Membership Library Yipeng Wang
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 1/7] member: implement main API Yipeng Wang
2017-09-22 10:47 ` Thomas Monjalon
2017-09-25 14:15 ` De Lara Guarch, Pablo
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 2/7] member: implement HT mode Yipeng Wang
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 3/7] member: implement vBF mode Yipeng Wang
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 4/7] member: add AVX for HT mode Yipeng Wang
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 5/7] member: enable the library Yipeng Wang
2017-09-22 10:48 ` Thomas Monjalon
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 6/7] test/member: add functional and perf tests Yipeng Wang
2017-09-05 23:59 ` [dpdk-dev] [PATCH v3 7/7] doc: add membership documentation Yipeng Wang
2017-09-18 18:42 ` Mcnamara, John
2017-09-25 12:30 ` De Lara Guarch, Pablo
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 0/7] Add Membership Library Yipeng Wang
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 1/7] member: implement main API Yipeng Wang
2017-10-02 10:04 ` De Lara Guarch, Pablo
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 2/7] member: implement HT mode Yipeng Wang
2017-10-02 13:30 ` De Lara Guarch, Pablo
2017-10-03 1:18 ` Wang, Yipeng1
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 3/7] member: implement vBF mode Yipeng Wang
2017-10-02 15:44 ` De Lara Guarch, Pablo
2017-10-03 1:24 ` Wang, Yipeng1
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 4/7] member: add AVX for HT mode Yipeng Wang
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 5/7] member: enable the library Yipeng Wang
2017-10-02 15:47 ` De Lara Guarch, Pablo
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 6/7] test/member: add functional and perf tests Yipeng Wang
2017-10-02 16:20 ` De Lara Guarch, Pablo
2017-09-27 17:40 ` [dpdk-dev] [PATCH v4 7/7] doc: add membership documentation Yipeng Wang
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 0/7] Add Membership Library Yipeng Wang
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 1/7] member: implement main API Yipeng Wang
2017-10-03 8:42 ` De Lara Guarch, Pablo
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 2/7] member: implement HT mode Yipeng Wang
2017-10-03 8:47 ` De Lara Guarch, Pablo
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 3/7] member: implement vBF mode Yipeng Wang
2017-10-03 8:50 ` De Lara Guarch, Pablo
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 4/7] member: add AVX for HT mode Yipeng Wang
2017-10-03 9:01 ` De Lara Guarch, Pablo
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 5/7] member: enable the library Yipeng Wang
2017-10-03 9:04 ` De Lara Guarch, Pablo
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 6/7] test/member: add functional and perf tests Yipeng Wang
2017-10-03 9:07 ` De Lara Guarch, Pablo
2017-10-03 4:31 ` [dpdk-dev] [PATCH v5 7/7] doc: add membership documentation Yipeng Wang
2017-10-03 9:08 ` De Lara Guarch, Pablo
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 0/7] Add Membership Library Yipeng Wang
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 1/7] member: implement main API Yipeng Wang
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 2/7] member: implement HT mode Yipeng Wang
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 3/7] member: implement vBF mode Yipeng Wang
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 4/7] member: add AVX for HT mode Yipeng Wang
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 5/7] member: enable the library Yipeng Wang
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 6/7] test/member: add functional and perf tests Yipeng Wang
2017-10-04 3:12 ` [dpdk-dev] [PATCH v6 7/7] doc: add membership documentation Yipeng Wang
2017-10-04 13:44 ` Mcnamara, John
2017-10-08 22:14 ` [dpdk-dev] [PATCH v6 0/7] Add Membership Library Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1504396174.21816.0.camel@gmail.com \
--to=luca.boccassi@gmail.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
--cc=yipeng1.wang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).