From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 4E902133F for ; Wed, 15 Mar 2017 22:11:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=intel.com; i=@intel.com; q=dns/txt; s=intel; t=1489612319; x=1521148319; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=KTRMDH1RKY675ZfkXz/ggnxvlKDTiRr6mFqOTyDoJAE=; b=lIq70opYINOqPmMyeK4+tw2ZfvQe4BNT4kDMbRYcVKxJoVDBXnqHomSX P7is1FG4o10njkmd3bosHlkerSHLQQ==; Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 Mar 2017 14:11:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.36,170,1486454400"; d="scan'208";a="67762701" Received: from irsmsx107.ger.corp.intel.com ([163.33.3.99]) by orsmga004.jf.intel.com with ESMTP; 15 Mar 2017 14:11:57 -0700 Received: from irsmsx108.ger.corp.intel.com ([169.254.11.173]) by IRSMSX107.ger.corp.intel.com ([169.254.10.3]) with mapi id 14.03.0248.002; Wed, 15 Mar 2017 21:11:56 +0000 From: "Dumitrescu, Cristian" To: Thomas Monjalon CC: "De Lara Guarch, Pablo" , "Singh, Jasvinder" , "dev@dpdk.org" , "Doherty, Declan" Thread-Topic: [dpdk-dev] [PATCH v2 1/2] librte_net: add crc init and compute APIs Thread-Index: AQHSnckaDX5U4LP+bkC/zV/0yuXpIKGWXPgg Date: Wed, 15 Mar 2017 21:11:55 +0000 Message-ID: <3EB4FA525960D640B5BDFFD6A3D8912652760A86@IRSMSX108.ger.corp.intel.com> References: <1487969657-172541-2-git-send-email-jasvinder.singh@intel.com> <6450918.dWRLMxV3EZ@xps13> <3EB4FA525960D640B5BDFFD6A3D89126527609A4@IRSMSX108.ger.corp.intel.com> <9340287.qWvXv5QBAa@xps13> In-Reply-To: <9340287.qWvXv5QBAa@xps13> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZjliOTU4MGItNWQwYS00MWI0LTlhNTItZmU0M2E2M2Y4OGYzIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX0lDIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE2LjUuOS4zIiwiVHJ1c3RlZExhYmVsSGFzaCI6ImVPUEt6Ynk4VERNbkRvVGE2KzFaRzJJcFVQbFZHekRucnV0dWcrYXR2RTA9In0= x-ctpclassification: CTP_IC x-originating-ip: [163.33.239.180] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v2 1/2] librte_net: add crc init and compute APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Mar 2017 21:11:59 -0000 > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Wednesday, March 15, 2017 8:16 PM > To: Dumitrescu, Cristian > Cc: De Lara Guarch, Pablo ; Singh, > Jasvinder ; dev@dpdk.org; Doherty, Declan > > Subject: Re: [dpdk-dev] [PATCH v2 1/2] librte_net: add crc init and compu= te > APIs >=20 > 2017-03-15 19:03, Dumitrescu, Cristian: > > ... > > > > > > > > > I think it should be in librte_hash. > > > > > > > > > > > > > > Please check lib/librte_hash/rte_hash_crc.h > > > > > > > > > > > > Is it good to include payload crc calculation in hash library a= s I see all > > > hash > > > > > related functionality there? > > > > > > > > > > I think yes. Pablo? > > > > > > > > I think this doesn't belong in the hash library. These new function= s > calculate > > > CRC, but not as a hash function. > > > > > > Can't we say that a CRC is a hash? What is a hash? > > > A function generating the same output bytes from given input bytes. > > > > > > I think you must separate hash functions and hash table management. > > > > > > > The fact that CRC32 instruction is opportunistically used to compute a = hash > digest/signature for load balancing (affinity-based) or hash tables (flow > tables, ARP cache, etc) does not mean that all the code that uses CRC32 > instruction should be placed in librte_hash. > > > > The purpose of the hash functions in librte_hash is to compute a > digest/signature for a given array of bytes (the key) as fast as possible= . Any > hash function that produces a hash signature with good uniform distributi= on > in a small amount of cycles belongs here, including those opportunistical= ly > using specialized CPU instructions such as CRC32 (or XOR, AESNI, etc). > > > > The code proposed in this patch is used to compute packet fields for > various protocols that have a CRC field, such as FCS of Ethernet frames, = etc. > according to the relevant standard (IEEE 802, others). It is an utility t= o be used > for implementing protocol-level functionality for various protocols from = the > network stack, similar to e.g. IP or UDP checksum. If we were to add an I= P or > UDCP checksum calculator, would you put it in librte_hash? > > > > The code from this patch is never going to be used to compute a fast > signature/digest. Typically this CRC is computed over the entire frame/pa= cket > rather than just selected fields from the packet representing the applica= tion- > specific flow key. Also note that the signature produced by CRC32 hash > function from librte_hash is actually not the correct Cyclic Redundancy C= heck > of that array of bytes (or, for math guys, of the associated polynomial),= it is > just a partial/intermediate value. > > > > Therefore, I suggest placing this code into: librte_ether (given that i= t can be > used to compute Ethernet FCS), or librte_net, or librte_crc. Definitely n= ot in > librte_hash. >=20 > I agree with you Cristian that the protocol layer must be in librte_net. > But I think most of this patch is not protocol level. Nope, this is the true CRC computed over entire protocol header and/or payl= oad. Similar to to IPv4/UDP/TCP checksum. The only reason for computing it = is because the protocol specs require it for data integrity checks, nothing= to do with our signature for load balancing/hash tables. More details on covered protocols from a reliable source :) [1]: CRC-32 (polynomial =3D 0x04C11DB7): used for HDLC, ANSI X3.66, ITU-T V.42,= Ethernet, Serial ATA, MPEG-2, PKZIP, Gzip, Bzip2, PNG, many others CRC-16-CCITT (polynomial =3D 0x1021): used for X.25, V.41, HDLC FCS, XMODE= M, Bluetooth, PACTOR, SD, DigRF, many others [1] Wikipedia: https://en.wikipedia.org/wiki/Cyclic_redundancy_check#Standa= rds_and_common_use > I think you agree with me that the code computing a > "digest/signature for a given array of bytes" must go to librte_hash? Yes, but this is the true protocol-level CRC, not a digest/signature. Non-cryptographic hash digest/signature: -computed over selected packet fields (flow key) for load balancing (affini= ty scheme) or hash table; key size typical range: 8 .. 192 bytes -required by application requirements (such as flow packet ordering preserv= ation), not by protocol standards -has uniform distribution -requires small amount of cycles to compute -used as meta-data, not written in the packet -can be opportunistically generated using specialized CPU instructions, suc= h as CRC32 (or XEOR, or AESNI); in this case, it is a partial/intermediate = value, not the correct CRC of the array of bytes Protocol CRC: -computed over entire packet header and/or payload -protocol overhead (required by standards) -computational cost is typically big and proportional with the packet lengt= h; packet length typical range: 64 .. 1514 .. 9K -written in the packet (by the application SW or by the HW)