From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7C57AA0032; Fri, 16 Sep 2022 04:30:48 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 14CFC4021D; Fri, 16 Sep 2022 04:30:48 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by mails.dpdk.org (Postfix) with ESMTP id E1CAC40156 for ; Fri, 16 Sep 2022 04:30:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1663295446; x=1694831446; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=iKnriT+0rlrq/G13LIplDHWi3tIdKHS+zJZAYyV/z4Y=; b=V1CaVulEcggMSqTB5uIXRP0UWyHujBQqJ8s6E5ZABCc3rEpcCNkPxkEQ M53zvscHkNMkltRJ0moktgwGXwFBGE6tO+vIFjh6o3j67eAKsBZqlHVy/ nahxqD5Vh9roSEpJbeTqRwJqMeqKDnNoDTyC0rbbWLyUXi3G8dwUUHym5 3Zv2iKOxYhV/2iiPVZ0zEoQ5D3chzqzlmIsLTuwUz5P/Q8wLlQiT9mQ8s EZD2Ih1KE+1dGgzdiEqgAIAM2Ig217oZ2dCKUIeADGLBW1OcCYAU2F+jd uyP4Y+1ot1DJV19iLTvx2PAkyxTtJVY/aU92NpSo56DlOBX3USxput8Yw g==; X-IronPort-AV: E=McAfee;i="6500,9779,10471"; a="297616342" X-IronPort-AV: E=Sophos;i="5.93,319,1654585200"; d="scan'208";a="297616342" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Sep 2022 19:30:36 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,319,1654585200"; d="scan'208";a="685967292" Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83]) by fmsmga004.fm.intel.com with ESMTP; 15 Sep 2022 19:30:36 -0700 Received: from fmsmsx607.amr.corp.intel.com (10.18.126.87) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 15 Sep 2022 19:30:36 -0700 Received: from fmsmsx612.amr.corp.intel.com (10.18.126.92) by fmsmsx607.amr.corp.intel.com (10.18.126.87) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 15 Sep 2022 19:30:35 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx612.amr.corp.intel.com (10.18.126.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31 via Frontend Transport; Thu, 15 Sep 2022 19:30:35 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.168) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.31; Thu, 15 Sep 2022 19:30:35 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hMHt8oUHt3Jl3LbyzPZ9TPU3fqckTrayLjVgNAppZbK5HUU3ZppdgxuAnvCcKlcBOSDl7ZgL1al86WmW0d6u0VoS53m9HG7MSvCW6ucnxxRl4iaIrV6gexE7Lc71rziammPJTUijShHgp7YVYwXH9NRlPzPXo39t/C236qx51mNCyMcIvUSJNgduO8r64/6lem8pwqL5SVfdiG6Wvhyx8DedXOqs/cuHsI2g5Cas4LRtBYRPTvzVk6EAcgXx7MQGLY/gPk/y2N0x0/GP+Y7MsSMiIklY2rfEKFupH03PV+TP+Pdugb8fIwukOrM7GJcwk7pjqpp2OoQUdgpffX9wCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eX64Xpu9oGIOjFt+iqX5xqTYzMxW0XiBkCro3sQDvP8=; b=HWejbXxMM/KYHyTi1GWxkpiT+pbw6leG7bk/Pw3ZEN7hI7VHW7oyDXoG1TjwOVTGpdA8Vq8KoQ/U7vIgq85ATrPZdcXooMqp6nMubVSkduE5FjqnsMjNVqVIRvIrPlE3tb570ViByKXdBpstVx+H2uDXmBdDmA0VG2sxWWbJLblpMOo5XCsj1fCBNJN1LR3up7t+tSo+egaLQXSwdLD+3FQH8+AmHeYxX4RlPfnwAdDnoEciW9JN29ANdhAOATGb6ikspXmSzyfLvtj5lrkYgNEY0RhDw2nHs9kr4He8PY0leM42ZwmHHIJfpJDCEYNRvI4d6gDS6ikUgoQNJQuTiQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from MW4PR11MB5774.namprd11.prod.outlook.com (2603:10b6:303:182::16) by SA2PR11MB5004.namprd11.prod.outlook.com (2603:10b6:806:112::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5632.15; Fri, 16 Sep 2022 02:30:27 +0000 Received: from MW4PR11MB5774.namprd11.prod.outlook.com ([fe80::2545:4b2f:7d0b:957d]) by MW4PR11MB5774.namprd11.prod.outlook.com ([fe80::2545:4b2f:7d0b:957d%7]) with mapi id 15.20.5612.022; Fri, 16 Sep 2022 02:30:27 +0000 From: "Rong, Leyi" To: "Richardson, Bruce" CC: "ferruh.yigit@xilinx.com" , "david.marchand@redhat.com" , "suanmingm@nvidia.com" , "Wang, Yipeng1" , "zaoxingliu@gmail.com" , "Gobriel, Sameh" , "dev@dpdk.org" Subject: RE: [PATCH v4 1/2] member: implement NitroSketch mode Thread-Topic: [PATCH v4 1/2] member: implement NitroSketch mode Thread-Index: AQHYyQO2OaaTkS9/XUKzdhAxENw/0K3giHQAgADMXUA= Date: Fri, 16 Sep 2022 02:30:27 +0000 Message-ID: References: <20220810074518.1695013-1-leyi.rong@intel.com> <20220915130342.1497512-1-leyi.rong@intel.com> <20220915130342.1497512-2-leyi.rong@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.6.500.17 authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: MW4PR11MB5774:EE_|SA2PR11MB5004:EE_ x-ms-office365-filtering-correlation-id: c05755c1-9fd2-49e4-2624-08da978b6afc x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: TeNNu6YzFUNsFutkABzBdjEwR8GIjtAmvQBRyMCOrD+HzpMQUh9qYotEvnlFVo3FV0Hqa4mP2kLUZQbnvg3ItTspK4ynz53C8rSjmO3+fnrUZdTXuiLb96baz79Dryi0EOcJN+LCvMCOWINEOGrpww9ew55RumFC+ngSXF7qh6fy9H7ESFcdAMI6b7LZzyD+7ChUa81McEVXk0KpLARD7dF4Dqn6wjHEV4uwpwOURYWyASA4gTqzcaR7MNpFVnlRxUqlhB67/6DEe4iv81Pv1EyrDWjUq0Q5W/zNL8hCQofB9eehDF4kbmDmX40tgeL8q4EtXpwSB027KaFWFTQlJrJrk48HPSqVMiJ80mKSX/kVY6kTKrsXE3ttpNX4dSAC6+i/8FwE3c9L11OkjNNalUqMDfoh+i+Kb1JJhkmRr58T0aWsgzA7qET2bDW8+1XwIz9zXdDUz9iSWe1WlhiWGt63gt0JiNub61XZs6Se+eZNz9ayZyN4VRYDkvRS3JjINS73Lmhym6kCYHeg/LF+KM5Arz0AheyaRoeRXhIDFWTGbabGjVwNx4uVOD4wlHxACCIocjYMCCMEkJKZzrmOGsYFeknGSLMG/rotibsVsfUru8kuqdI0rY6vhXtn9XyBSIPVpOvWqTbdV8zM/q2DHnUaBv65LnpGx0Wa15UKNjnICgEm72+CUj4+yvUbja4PSswnsWzIia9jh9eGsQ+6hftGtydrbF9W6iyLa8fu7nod/m7Wr6fPRyrdcQ9RPZZD8l6SE12ZXANs2gXcOR3fdFenlqkVdcMB+h0xeIx38Ug= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MW4PR11MB5774.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(346002)(39860400002)(376002)(136003)(396003)(366004)(451199015)(86362001)(41300700001)(5660300002)(52536014)(8676002)(6862004)(76116006)(33656002)(8936002)(4326008)(64756008)(66446008)(66476007)(9686003)(26005)(83380400001)(316002)(54906003)(7696005)(6636002)(66946007)(186003)(66556008)(38070700005)(2906002)(6506007)(53546011)(966005)(55016003)(122000001)(38100700002)(478600001)(71200400001)(82960400001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?iKXoL4sb+aOKvjiPb8zi7q0/IVC19hQHuDaxm7qfIMlQ7u+Rdi8e62kSditU?= =?us-ascii?Q?jDHX7E1X5z21aQpLlSSHJCHFh2m5jgrXORlQXtgnQ5BBN18dDwg2EkQSImOc?= =?us-ascii?Q?F3rW8OyslEtuB8/ZlJIEEWi+540dOHq2h8Ax1e30mhVInStlsNLrvZHGyU62?= =?us-ascii?Q?c552LYTJa5bMQ1cSzsB8Sfh/UC1K8J00gKEdz1Q2aPUczFW1uRKIpMV6r732?= =?us-ascii?Q?iFGF8dPbH+24g4lGBHImbqtZ7cpsQszbS1VUhpY5Ac/WCvC2p1AuKfZhzWDi?= =?us-ascii?Q?qnyz5/98MLf6edj81laLVUpeNyLT06trsJ9rfH3gO5g+BUIUkPP1mos4Nfxr?= =?us-ascii?Q?BF8UA15l+u23I31PPZp2BMxM0m0lOityND1hUD18i8tXIZ7dCtVe2pX2hNtl?= =?us-ascii?Q?33Pl4vontZ4jfaSfvssbHWRWpwgZQoa6y0SbvjUmKa5vGALL2Lr5EOn9XePi?= =?us-ascii?Q?zJ3sNog2oGnWlucrkCzHOuySgc0x6m1VhUf8HdPZCRjjuEOiTqpNa6ItbVeE?= =?us-ascii?Q?0qw55u34xw9QE9yHDKsoDMmNWJyhSFhNI0jBnqb4egoX3L+GhJP/Sqtk50iL?= =?us-ascii?Q?dUI0BeDjjgwT2E7mFIJAbUz8ztlwIbLhLzh786e+7w063rvVBpqItodG0HH7?= =?us-ascii?Q?y5rIe0lXXl/TKWj000k6D0OSr2bcX6VWlD1GvA7AJjTADczy0jvEH0FUM1oF?= =?us-ascii?Q?5wHp8nIOA/q59lVjW9zrpfxf7MUizLQXwL4eODBy9v+08R+W2nyDjqQfN81t?= =?us-ascii?Q?bg0dRsZoNQmFlED+kv1QrIILtoit8YNB8b7MBiEpFWjW1OTBltTfglMP3oSx?= =?us-ascii?Q?tEAodDfE5e8JTxaFShAR98Xqqmz4vdLmOjVzage1bjsdOUmyM/s0SK/lwxIp?= =?us-ascii?Q?EVIqjqpC7ULvlxY6t2ImdZxK/GrePzNXlcYU+iha8bFtV5+zDXTXpvb9Z+tG?= =?us-ascii?Q?k2AzMB1eo+wqIfmmMC8pwC2PgWK2Cw+hYdDg9JZyao+WqOYVTId4BxRYkoVw?= =?us-ascii?Q?2x6i4Xf+e78cnARx3CAx2dPWA3rixL32auDiPcJS5vdr/Eaqo5Y+hLiDLrkR?= =?us-ascii?Q?RAlNtrFD1eqwffee6eKy1iu5QTv+dqoUokHzs42D1DwubiNFd5U2j07WjYUE?= =?us-ascii?Q?npBkk24i7oJ/aDKjqk35l/Lw10kJGHbf4BlVrAa6CDfLDH8Je4y+dYa4CrJw?= =?us-ascii?Q?VIkJOfgmdpZHPiCzX0kM73aFcWi7Ge8vnTc5EX+EqaHCtODeJYxjELJfuhjB?= =?us-ascii?Q?YWgPEBOLlDf+S5tDKS/Ao6mLTVFlYWeUGlDOv/W3LaCWSuElWda3g6Xsgurx?= =?us-ascii?Q?uq7X5bpAq0P43OFe1AIb5Dow+SyttOYj0A6ESwTp2XMuoA5t+Q+A3rzAVGG0?= =?us-ascii?Q?DU1vGikEj1mGbD2hkdRwt2UFlBFql+cP+UpGm6kFw+jAXI/Dzrr9PLzS8JEV?= =?us-ascii?Q?/mvGfEozkjEW9M+gSLavjvB/zBLpby45XlzMhDwiyBFc7FPoyUw5OvtLsWb5?= =?us-ascii?Q?PkySkbfjSXL+06V+CzFkJUBc8XDFya1azIGNE7zWgyEkT7I7dnE+MU75W7bS?= =?us-ascii?Q?pakMLEwnjIdAFY8su93C/E0T/mKkZlAbvYYvx6vU?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB5774.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: c05755c1-9fd2-49e4-2624-08da978b6afc X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2022 02:30:27.7191 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: m9OO15D2qQF+RjMmhWQErMJsUo4cuNnGGpnM4wMqjpVZklE7JCFC1OV9L5YtfbNAQdcLDUW0D9fDe6A08SdEnQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA2PR11MB5004 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Bruce, The comments will be addressed in v5, thank you! > -----Original Message----- > From: Richardson, Bruce > Sent: Thursday, September 15, 2022 10:13 PM > To: Rong, Leyi > Cc: ferruh.yigit@xilinx.com; david.marchand@redhat.com; > suanmingm@nvidia.com; Wang, Yipeng1 ; > zaoxingliu@gmail.com; Gobriel, Sameh ; > dev@dpdk.org > Subject: Re: [PATCH v4 1/2] member: implement NitroSketch mode >=20 > On Thu, Sep 15, 2022 at 09:03:41PM +0800, Leyi Rong wrote: > > Sketching algorithm provide high-fidelity approximate measurements and > > appears as a promising alternative to traditional approaches such as > > packet sampling. > > > > NitroSketch [1] is a software sketching framework that optimizes > > performance, provides accuracy guarantees, and supports a variety of > > sketches. > > > > This commit adds a new data structure called sketch into membership > > library. This new data structure is an efficient way to profile the > > traffic for heavy hitters. Also use min-heap structure to maintain the > > top-k flow keys. > > > > [1] Zaoxing Liu, Ran Ben-Basat, Gil Einziger, Yaron Kassner, Vladimir > > Braverman, Roy Friedman, Vyas Sekar, "NitroSketch: Robust and General > > Sketch-based Monitoring in Software Switches", in ACM SIGCOMM 2019. > > https://dl.acm.org/doi/pdf/10.1145/3341302.3342076 > > > > Signed-off-by: Alan Liu > > Signed-off-by: Yipeng Wang > > Signed-off-by: Leyi Rong > > --- >=20 > Couple of comments below on the meson.build file. >=20 > /Bruce >=20 > > lib/member/meson.build | 42 +- > > lib/member/rte_member.c | 75 ++++ > > lib/member/rte_member.h | 154 ++++++- > > lib/member/rte_member_heap.h | 424 ++++++++++++++++++ > > lib/member/rte_member_sketch.c | 594 ++++++++++++++++++++++++++ > > lib/member/rte_member_sketch.h | 97 +++++ > > lib/member/rte_member_sketch_avx512.c | 70 +++ > > lib/member/rte_member_sketch_avx512.h | 35 ++ > > lib/member/rte_xxh64_avx512.h | 117 +++++ > > lib/member/version.map | 9 + > > 10 files changed, 1613 insertions(+), 4 deletions(-) create mode > > 100644 lib/member/rte_member_heap.h create mode 100644 > > lib/member/rte_member_sketch.c create mode 100644 > > lib/member/rte_member_sketch.h create mode 100644 > > lib/member/rte_member_sketch_avx512.c > > create mode 100644 lib/member/rte_member_sketch_avx512.h > > create mode 100644 lib/member/rte_xxh64_avx512.h > > > > diff --git a/lib/member/meson.build b/lib/member/meson.build index > > e06fddc240..3df0fbc7a6 100644 > > --- a/lib/member/meson.build > > +++ b/lib/member/meson.build > > @@ -7,6 +7,46 @@ if is_windows > > subdir_done() > > endif > > > > -sources =3D files('rte_member.c', 'rte_member_ht.c', > > 'rte_member_vbf.c') > > +sources =3D files('rte_member.c', 'rte_member_ht.c', > > +'rte_member_vbf.c', 'rte_member_sketch.c') >=20 > At this point, with 4 entries I'd suggest reworking this to a vertical li= st. >=20 > sources =3D files( > 'rte_member.c', > .... > ) >=20 > > headers =3D files('rte_member.h') > > deps +=3D ['hash'] > > +includes +=3D include_directories('../hash', '../ring') > > + >=20 > Use dependencies rather than include-paths. Having a dependency on hash > means that your code will be compiled with the hash include paths already > present. Similarly you can add "ring" to the deps variable and drop this = line. >=20 > > +# compile AVX512 version if: > > +# we are building 64-bit binary AND binutils can generate proper code >=20 > I think this comment is no longer necessary. The condition itself is clea= r. > The following comment can be kept though as it explains what is being don= e at a > higher level than the code. >=20 > > +if dpdk_conf.has('RTE_ARCH_X86_64') and binutils_ok > > + # compile AVX512 version if either: > > + # a. we have AVX512 supported in minimum instruction set > > + # baseline > > + # b. it's not minimum instruction set, but supported by > > + # compiler > > + # > > + # in former case, just add avx512 C file to files list > > + # in latter case, compile c file to static lib, using correct > > + # compiler flags, and then have the .o file from static lib > > + # linked into main lib. > > + > > + #check if all required flags already enabled > > + sketch_avx512_flags =3D ['__AVX512F__', '__AVX512DQ__', > > + '__AVX512IFMA__'] > > + > > + sketch_avx512_on =3D true > > + foreach f:sketch_avx512_flags > > + if cc.get_define(f, args: machine_args) =3D=3D '' > > + sketch_avx512_on =3D false > > + endif > > + endforeach > > + > > + if sketch_avx512_on =3D=3D true > > + cflags +=3D ['-DCC_AVX512_SUPPORT'] > > + sources +=3D files('rte_member_sketch_avx512.c') > > + elif cc.has_multi_arguments('-mavx512f', '-mavx512dq', '-mavx512if= ma') > > + sketch_avx512_tmp =3D static_library('sketch_avx512_tmp', > > + 'rte_member_sketch_avx512.c', > > + include_directories: includes, > > + dependencies: static_rte_eal, > > + c_args: cflags + > > + ['-mavx512f', '-mavx512dq', '-mavx512ifma']) > > + objs +=3D > sketch_avx512_tmp.extract_objects('rte_member_sketch_avx512.c') > > + cflags +=3D ['-DCC_AVX512_SUPPORT'] >=20 > Logic looks correct here now, but beware of your indentation. You are mix= ing > tabs and spaces. Meson files should have spaces only, no tabs. >=20 > > + endif > > +endif