From: "Dumitrescu, Cristian" <cristian.dumitrescu@intel.com>
To: Thomas Monjalon <thomas.monjalon@6wind.com>
Cc: "De Lara Guarch, Pablo" <pablo.de.lara.guarch@intel.com>,
"Singh, Jasvinder" <jasvinder.singh@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>,
"Doherty, Declan" <declan.doherty@intel.com>
Subject: Re: [dpdk-dev] [PATCH v2 1/2] librte_net: add crc init and compute APIs
Date: Wed, 15 Mar 2017 21:11:55 +0000 [thread overview]
Message-ID: <3EB4FA525960D640B5BDFFD6A3D8912652760A86@IRSMSX108.ger.corp.intel.com> (raw)
In-Reply-To: <9340287.qWvXv5QBAa@xps13>
> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> Sent: Wednesday, March 15, 2017 8:16 PM
> To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>
> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Singh,
> Jasvinder <jasvinder.singh@intel.com>; dev@dpdk.org; Doherty, Declan
> <declan.doherty@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v2 1/2] librte_net: add crc init and compute
> APIs
>
> 2017-03-15 19:03, Dumitrescu, Cristian:
> > ... <snip>
> >
> > > > > > > I think it should be in librte_hash.
> > > > > > >
> > > > > > > Please check lib/librte_hash/rte_hash_crc.h
> > > > > >
> > > > > > Is it good to include payload crc calculation in hash library as I see all
> > > hash
> > > > > related functionality there?
> > > > >
> > > > > I think yes. Pablo?
> > > >
> > > > I think this doesn't belong in the hash library. These new functions
> calculate
> > > CRC, but not as a hash function.
> > >
> > > Can't we say that a CRC is a hash? What is a hash?
> > > A function generating the same output bytes from given input bytes.
> > >
> > > I think you must separate hash functions and hash table management.
> > >
> >
> > The fact that CRC32 instruction is opportunistically used to compute a hash
> digest/signature for load balancing (affinity-based) or hash tables (flow
> tables, ARP cache, etc) does not mean that all the code that uses CRC32
> instruction should be placed in librte_hash.
> >
> > The purpose of the hash functions in librte_hash is to compute a
> digest/signature for a given array of bytes (the key) as fast as possible. Any
> hash function that produces a hash signature with good uniform distribution
> in a small amount of cycles belongs here, including those opportunistically
> using specialized CPU instructions such as CRC32 (or XOR, AESNI, etc).
> >
> > The code proposed in this patch is used to compute packet fields for
> various protocols that have a CRC field, such as FCS of Ethernet frames, etc.
> according to the relevant standard (IEEE 802, others). It is an utility to be used
> for implementing protocol-level functionality for various protocols from the
> network stack, similar to e.g. IP or UDP checksum. If we were to add an IP or
> UDCP checksum calculator, would you put it in librte_hash?
> >
> > The code from this patch is never going to be used to compute a fast
> signature/digest. Typically this CRC is computed over the entire frame/packet
> rather than just selected fields from the packet representing the application-
> specific flow key. Also note that the signature produced by CRC32 hash
> function from librte_hash is actually not the correct Cyclic Redundancy Check
> of that array of bytes (or, for math guys, of the associated polynomial), it is
> just a partial/intermediate value.
> >
> > Therefore, I suggest placing this code into: librte_ether (given that it can be
> used to compute Ethernet FCS), or librte_net, or librte_crc. Definitely not in
> librte_hash.
>
> I agree with you Cristian that the protocol layer must be in librte_net.
> But I think most of this patch is not protocol level.
Nope, this is the true CRC computed over entire protocol header and/or payload. Similar to to IPv4/UDP/TCP checksum. The only reason for computing it is because the protocol specs require it for data integrity checks, nothing to do with our signature for load balancing/hash tables.
More details on covered protocols from a reliable source :) [1]:
CRC-32 (polynomial = 0x04C11DB7): used for HDLC, ANSI X3.66, ITU-T V.42, Ethernet, Serial ATA, MPEG-2, PKZIP, Gzip, Bzip2, PNG, many others
CRC-16-CCITT (polynomial = 0x1021): used for X.25, V.41, HDLC FCS, XMODEM, Bluetooth, PACTOR, SD, DigRF, many others
[1] Wikipedia: https://en.wikipedia.org/wiki/Cyclic_redundancy_check#Standards_and_common_use
> I think you agree with me that the code computing a
> "digest/signature for a given array of bytes" must go to librte_hash?
Yes, but this is the true protocol-level CRC, not a digest/signature.
Non-cryptographic hash digest/signature:
-computed over selected packet fields (flow key) for load balancing (affinity scheme) or hash table; key size typical range: 8 .. 192 bytes
-required by application requirements (such as flow packet ordering preservation), not by protocol standards
-has uniform distribution
-requires small amount of cycles to compute
-used as meta-data, not written in the packet
-can be opportunistically generated using specialized CPU instructions, such as CRC32 (or XEOR, or AESNI); in this case, it is a partial/intermediate value, not the correct CRC of the array of bytes
Protocol CRC:
-computed over entire packet header and/or payload
-protocol overhead (required by standards)
-computational cost is typically big and proportional with the packet length; packet length typical range: 64 .. 1514 .. 9K
-written in the packet (by the application SW or by the HW)
next prev parent reply other threads:[~2017-03-15 21:11 UTC|newest]
Thread overview: 69+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-02-24 20:54 [dpdk-dev] [PATCH 0/2] librte_net: add crc computation support Jasvinder Singh
2017-02-24 20:54 ` [dpdk-dev] [PATCH 1/2] librte_net: add crc init and compute APIs Jasvinder Singh
2017-02-28 12:08 ` [dpdk-dev] [PATCH v2 0/2] librte_net: add crc computation support Jasvinder Singh
2017-02-28 12:08 ` [dpdk-dev] [PATCH v2 1/2] librte_net: add crc init and compute APIs Jasvinder Singh
2017-02-28 12:15 ` Jerin Jacob
2017-03-01 18:46 ` Thomas Monjalon
2017-03-02 13:03 ` Singh, Jasvinder
2017-03-06 15:27 ` Thomas Monjalon
2017-03-08 11:08 ` De Lara Guarch, Pablo
2017-03-15 17:35 ` Thomas Monjalon
2017-03-15 19:03 ` Dumitrescu, Cristian
2017-03-15 20:15 ` Thomas Monjalon
2017-03-15 21:11 ` Dumitrescu, Cristian [this message]
2017-03-15 19:09 ` Dumitrescu, Cristian
2017-03-12 21:33 ` [dpdk-dev] [PATCH v3 0/2] librte_net: add crc computation support Jasvinder Singh
2017-03-12 21:33 ` [dpdk-dev] [PATCH v3 1/2] librte_net: add crc compute APIs Jasvinder Singh
2017-03-13 3:06 ` Ananyev, Konstantin
2017-03-13 9:05 ` Singh, Jasvinder
2017-03-20 19:29 ` [dpdk-dev] [PATCH v4 0/2] librte_net: add crc computation support Jasvinder Singh
2017-03-20 19:29 ` [dpdk-dev] [PATCH v4 1/2] librte_net: add crc compute APIs Jasvinder Singh
2017-03-21 14:45 ` [dpdk-dev] [PATCH v5 0/2] librte_net: add crc computation support Jasvinder Singh
2017-03-21 14:45 ` [dpdk-dev] [PATCH v5 1/2] librte_net: add crc compute APIs Jasvinder Singh
2017-03-28 18:04 ` De Lara Guarch, Pablo
2017-03-28 18:07 ` De Lara Guarch, Pablo
2017-03-28 19:21 ` Singh, Jasvinder
2017-03-29 12:42 ` [dpdk-dev] [PATCH v6 0/2] librte_net: add crc computation support Jasvinder Singh
2017-03-29 12:42 ` [dpdk-dev] [PATCH v6 1/2] librte_net: add crc compute APIs Jasvinder Singh
2017-03-29 16:14 ` De Lara Guarch, Pablo
2017-03-29 17:15 ` [dpdk-dev] [PATCH v7 0/2] librte_net: add crc computation support Jasvinder Singh
2017-03-29 17:15 ` [dpdk-dev] [PATCH v7 1/2] librte_net: add crc compute APIs Jasvinder Singh
2017-03-30 11:30 ` [dpdk-dev] [PATCH v8 0/2] librte_net: add crc computation support Jasvinder Singh
2017-03-30 11:30 ` [dpdk-dev] [PATCH v8 1/2] librte_net: add crc compute APIs Jasvinder Singh
2017-03-30 11:31 ` Ananyev, Konstantin
2017-03-30 12:06 ` Singh, Jasvinder
2017-03-30 14:40 ` Olivier Matz
2017-03-30 15:14 ` Singh, Jasvinder
2017-03-30 16:15 ` [dpdk-dev] [PATCH v9 0/3] librte_net: add crc computation support Jasvinder Singh
2017-03-30 16:15 ` [dpdk-dev] [PATCH v9 1/3] librte_net: add crc compute APIs Jasvinder Singh
2017-04-04 20:00 ` Thomas Monjalon
2017-04-05 14:58 ` [dpdk-dev] [PATCH v10 0/2] librte_net: add crc computation support Jasvinder Singh
2017-04-05 14:58 ` [dpdk-dev] [PATCH v10 1/2] librte_net: add crc compute APIs Jasvinder Singh
2017-04-05 17:49 ` Thomas Monjalon
2017-04-05 19:22 ` Singh, Jasvinder
2017-04-05 20:49 ` [dpdk-dev] [PATCH v11 0/2] librte_net: add crc computation support Jasvinder Singh
2017-04-05 20:49 ` [dpdk-dev] [PATCH v11 1/2] librte_net: add crc compute APIs Jasvinder Singh
2017-04-05 20:49 ` [dpdk-dev] [PATCH v11 2/2] test/test: add unit test for CRC computation Jasvinder Singh
2017-04-05 20:59 ` Thomas Monjalon
2017-04-05 21:00 ` [dpdk-dev] [PATCH v11 0/2] librte_net: add crc computation support Thomas Monjalon
2017-04-05 14:58 ` [dpdk-dev] [PATCH v10 2/2] test/test: add unit test for CRC computation Jasvinder Singh
2017-03-30 16:15 ` [dpdk-dev] [PATCH v9 2/3] " Jasvinder Singh
2017-03-30 16:15 ` [dpdk-dev] [PATCH v9 3/3] maintainers: add packet crc section and claim maintainership Jasvinder Singh
2017-04-04 19:55 ` Thomas Monjalon
2017-04-04 20:02 ` [dpdk-dev] [PATCH v9 0/3] librte_net: add crc computation support Thomas Monjalon
2017-04-05 8:34 ` Singh, Jasvinder
2017-04-05 9:01 ` Thomas Monjalon
2017-04-05 9:37 ` Richardson, Bruce
2017-04-05 12:52 ` Singh, Jasvinder
2017-03-30 11:30 ` [dpdk-dev] [PATCH v8 2/2] test/test: add unit test for CRC computation Jasvinder Singh
2017-03-29 17:15 ` [dpdk-dev] [PATCH v7 " Jasvinder Singh
2017-03-29 12:42 ` [dpdk-dev] [PATCH v6 " Jasvinder Singh
2017-03-29 16:12 ` De Lara Guarch, Pablo
2017-03-21 14:45 ` [dpdk-dev] [PATCH v5 " Jasvinder Singh
2017-03-28 19:23 ` De Lara Guarch, Pablo
2017-03-28 19:27 ` Singh, Jasvinder
2017-03-20 19:29 ` [dpdk-dev] [PATCH v4 2/2] app/test: " Jasvinder Singh
2017-03-21 7:14 ` Peng, Yuan
2017-03-12 21:33 ` [dpdk-dev] [PATCH v3 " Jasvinder Singh
2017-02-28 12:08 ` [dpdk-dev] [PATCH v2 " Jasvinder Singh
2017-02-24 20:54 ` [dpdk-dev] [PATCH " Jasvinder Singh
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3EB4FA525960D640B5BDFFD6A3D8912652760A86@IRSMSX108.ger.corp.intel.com \
--to=cristian.dumitrescu@intel.com \
--cc=declan.doherty@intel.com \
--cc=dev@dpdk.org \
--cc=jasvinder.singh@intel.com \
--cc=pablo.de.lara.guarch@intel.com \
--cc=thomas.monjalon@6wind.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).