Re: [PATCH v2 1/2] member: implement NitroSketch mode

[David Marchand <david.marchand@redhat.com>]

On Wed, Aug 31, 2022 at 8:47 AM Leyi Rong <leyi.rong@intel.com> wrote:
>
> Sketching algorithm provide high-fidelity approximate measurements and
> appears as a promising alternative to traditional approaches such as
> packet sampling.
>
> NitroSketch [1] is a software sketching framework that optimizes
> performance, provides accuracy guarantees, and supports a variety of
> sketches.
>
> This commit adds a new data structure called sketch into
> membership library. This new data structure is an efficient
> way to profile the traffic for heavy hitters. Also use min-heap
> structure to maintain the top-k flow keys.

http://mails.dpdk.org/archives/test-report/2022-August/304026.html

This patch adds new symbols in the API without going through the
experimental phase.
What is the rationale for skipping it?

>
> [1] Zaoxing Liu, Ran Ben-Basat, Gil Einziger, Yaron Kassner, Vladimir
> Braverman, Roy Friedman, Vyas Sekar, "NitroSketch: Robust and General
> Sketch-based Monitoring in Software Switches", in ACM SIGCOMM 2019.
> https://dl.acm.org/doi/pdf/10.1145/3341302.3342076
>
> Signed-off-by: Alan Liu <zaoxingliu@gmail.com>
> Signed-off-by: Yipeng Wang <yipeng1.wang@intel.com>
> Signed-off-by: Leyi Rong <leyi.rong@intel.com>
> ---
>  lib/member/meson.build                |  38 +-
>  lib/member/rte_member.c               |  75 ++++
>  lib/member/rte_member.h               | 151 ++++++-
>  lib/member/rte_member_heap.h          | 424 ++++++++++++++++++
>  lib/member/rte_member_sketch.c        | 594 ++++++++++++++++++++++++++
>  lib/member/rte_member_sketch.h        |  97 +++++
>  lib/member/rte_member_sketch_avx512.c |  69 +++
>  lib/member/rte_member_sketch_avx512.h |  36 ++
>  lib/member/rte_xxh64_avx512.h         | 117 +++++
>  lib/member/version.map                |   3 +
>  10 files changed, 1600 insertions(+), 4 deletions(-)
>  create mode 100644 lib/member/rte_member_heap.h
>  create mode 100644 lib/member/rte_member_sketch.c
>  create mode 100644 lib/member/rte_member_sketch.h
>  create mode 100644 lib/member/rte_member_sketch_avx512.c
>  create mode 100644 lib/member/rte_member_sketch_avx512.h
>  create mode 100644 lib/member/rte_xxh64_avx512.h
>
> diff --git a/lib/member/meson.build b/lib/member/meson.build
> index e06fddc240..9b3418c25c 100644
> --- a/lib/member/meson.build
> +++ b/lib/member/meson.build
> @@ -7,6 +7,42 @@ if is_windows
>      subdir_done()
>  endif
>
> -sources = files('rte_member.c', 'rte_member_ht.c', 'rte_member_vbf.c')
> +sources = files('rte_member.c', 'rte_member_ht.c', 'rte_member_vbf.c', 'rte_member_sketch.c')
>  headers = files('rte_member.h')
>  deps += ['hash']
> +includes += include_directories('../hash', '../ring')
> +
> +# compile AVX512 version if:
> +# we are building 64-bit binary AND binutils can generate proper code
> +if dpdk_conf.has('RTE_ARCH_X86_64') and binutils_ok
> +    # compile AVX512 version if either:
> +    # a. we have AVX512 supported in minimum instruction set
> +    #    baseline
> +    # b. it's not minimum instruction set, but supported by
> +    #    compiler
> +    #
> +    # in former case, just add avx512 C file to files list
> +    # in latter case, compile c file to static lib, using correct
> +    # compiler flags, and then have the .o file from static lib
> +    # linked into main lib.
> +    sketch_avx512_cpu_support = (
> +        cc.get_define('__AVX512F__', args: machine_args) != ''
> +    )
> +
> +    if sketch_avx512_cpu_support == true
> +       cflags += ['-DCC_AVX512_SUPPORT']
> +       if cc.has_multi_arguments('-mavx512f', '-mavx512dq', '-mavx512ifma')
> +           cflags += ['-mavx512f', '-mavx512dq', '-mavx512ifma']

Pushing those flags in the cflags is probably wrong, as the rest of
the library objects will be compiled with those AVX512 flags.
If later this library code is run on a non supporting AVX512 system,
it will trigger a runtime error.


Please look at how other libraries integrated AVX512.
Thanks.

-- 
David Marchand