DPDK patches and discussions
 help / color / mirror / Atom feed
From: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
To: olivier.matz@6wind.com, sthemmin@microsoft.com,
	jerinj@marvell.com, bruce.richardson@intel.com,
	david.marchand@redhat.com, pbhagavatula@marvell.com,
	konstantin.ananyev@intel.com, yipeng1.wang@intel.com,
	drc@linux.vnet.ibm.com, honnappa.nagarahalli@arm.com
Cc: dev@dpdk.org, dharmik.thakkar@arm.com, ruifeng.wang@arm.com,
	gavin.hu@arm.com, nd@arm.com
Subject: [dpdk-dev] [PATCH v10 0/6] lib/ring: APIs to support custom element size
Date: Sat, 18 Jan 2020 13:32:41 -0600	[thread overview]
Message-ID: <20200118193247.43831-1-honnappa.nagarahalli@arm.com> (raw)
In-Reply-To: <20190906190510.11146-1-honnappa.nagarahalli@arm.com>

The current rte_ring hard-codes the type of the ring element to 'void *',
hence the size of the element is hard-coded to 32b/64b. Since the ring
element type is not an input to rte_ring APIs, it results in couple
of issues:

1) If an application requires to store an element which is not 64b, it
   needs to write its own ring APIs similar to rte_event_ring APIs. This
   creates additional burden on the programmers, who end up making
   work-arounds and often waste memory.
2) If there are multiple libraries that store elements of the same
   type, currently they would have to write their own rte_ring APIs. This
   results in code duplication.

This patch adds new APIs to support configurable ring element size.
The APIs support custom element sizes by allowing to define the ring
element to be a multiple of 32b.

The aim is to achieve same performance as the existing ring
implementation.

v10
 - Improved comments in test case files (Olivier)
 - Fixed possible memory leaks (Olivier)
 - Changed 'test_ring_with_exact_size' to use unaligned
   addresses (Konstantin)
 - Changed the commit message for eventdev (Jerin)

v9
 - Split 'test_ring_burst_bulk_tests' test case into 4 smaller
   functions to address clang compilation time issue.
 - Addressed compilation failure in Intel CI in the hash changes.

v8
 - Changed the 128b copy elements inline function to use 'memcpy'
   to generate unaligned load/store instructions for x86. Generic
   copy function results in performance drop. (Konstantin)
 - Changed the API type #defines to be more clear (Konstantin)
 - Removed the code duplication in performance tests (Konstantin)
 - Fixed memory leak, changed test macros to inline functions (Konstantin)
 - Changed functional tests to test for 20B ring element. Fixed
   a bug in 32b element copy code for enqueue/dequeue(ring size
   needs to be normalized for 32b).
 - Squashed the functional and performance tests in their
   respective single commits.

v7
 - Merged the test cases to test both legacy APIs and
   rte_ring_xxx_elem APIs without code duplication (Konstantin, Olivier)
 - Performance test cases are merged as well (Konstantin, Olivier)
 - Macros to copy elements are converted into inline functions (Olivier)
 - Added back the changes to hash and event libraries

v6
 - Labelled as RFC to indicate the better status
 - Added unit tests to test the rte_ring_xxx_elem APIs
 - Corrected 'macro based partial memcpy' (5/6) patch
 - Added Konstantin's method after correction (6/6)
 - Check Patch shows significant warnings and errors mainly due
   copying code from existing test cases. None of them are harmful.
   I will fix them once we have an agreement.

v5
 - Use memcpy for chunks of 32B (Konstantin).
 - Both 'ring_perf_autotest' and 'ring_perf_elem_autotest' are available
   to compare the results easily.
 - Copying without memcpy is also available in 1/3, if anyone wants to
   experiment on their platform.
 - Added other platform owners to test on their respective platforms.

v4
 - Few fixes after more performance testing

v3
 - Removed macro-fest and used inline functions
   (Stephen, Bruce)

v2
 - Change Event Ring implementation to use ring templates
   (Jerin, Pavan)


Honnappa Nagarahalli (6):
  test/ring: use division for cycle count calculation
  lib/ring: apis to support configurable element size
  test/ring: add functional tests for rte_ring_xxx_elem APIs
  test/ring: modify perf test cases to use rte_ring_xxx_elem APIs
  lib/hash: use ring with 32b element size to save memory
  eventdev: use custom element size ring for event rings

 app/test/test_ring.c                 | 1383 +++++++++++++-------------
 app/test/test_ring.h                 |  187 ++++
 app/test/test_ring_perf.c            |  476 +++++----
 lib/librte_eventdev/rte_event_ring.c |  147 +--
 lib/librte_eventdev/rte_event_ring.h |   45 +-
 lib/librte_hash/rte_cuckoo_hash.c    |   94 +-
 lib/librte_hash/rte_cuckoo_hash.h    |    2 +-
 lib/librte_ring/Makefile             |    3 +-
 lib/librte_ring/meson.build          |    4 +
 lib/librte_ring/rte_ring.c           |   41 +-
 lib/librte_ring/rte_ring.h           |    1 +
 lib/librte_ring/rte_ring_elem.h      | 1003 +++++++++++++++++++
 lib/librte_ring/rte_ring_version.map |    2 +
 13 files changed, 2279 insertions(+), 1109 deletions(-)
 create mode 100644 app/test/test_ring.h
 create mode 100644 lib/librte_ring/rte_ring_elem.h

-- 
2.17.1


  parent reply	other threads:[~2020-01-18 19:33 UTC|newest]

Thread overview: 173+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-28 14:46 [dpdk-dev] [PATCH 0/5] lib/ring: templates " Honnappa Nagarahalli
2019-08-28 14:46 ` [dpdk-dev] [PATCH 1/5] lib/ring: apis to support configurable " Honnappa Nagarahalli
2019-08-28 14:46 ` [dpdk-dev] [PATCH 2/5] lib/ring: add template to support different element sizes Honnappa Nagarahalli
2019-10-01 11:47   ` Ananyev, Konstantin
2019-10-02  4:21     ` Honnappa Nagarahalli
2019-10-02  8:39       ` Ananyev, Konstantin
2019-10-03  3:33         ` Honnappa Nagarahalli
2019-10-03 11:51           ` Ananyev, Konstantin
2019-10-03 12:27             ` Ananyev, Konstantin
2019-10-03 22:49               ` Honnappa Nagarahalli
2019-08-28 14:46 ` [dpdk-dev] [PATCH 3/5] tools/checkpatch: relax constraints on __rte_experimental Honnappa Nagarahalli
2019-08-28 14:46 ` [dpdk-dev] [PATCH 4/5] lib/ring: add ring APIs to support 32b ring elements Honnappa Nagarahalli
2019-08-28 14:46 ` [dpdk-dev] [PATCH 5/5] lib/hash: use ring with 32b element size to save memory Honnappa Nagarahalli
2019-08-28 15:12 ` [dpdk-dev] [PATCH 0/5] lib/ring: templates to support custom element size Jerin Jacob Kollanukkaran
2019-08-28 15:16 ` Pavan Nikhilesh Bhagavatula
2019-08-28 22:59   ` Honnappa Nagarahalli
2019-09-06 19:05 ` [dpdk-dev] [PATCH v2 0/6] " Honnappa Nagarahalli
2019-09-06 19:05   ` [dpdk-dev] [PATCH v2 1/6] lib/ring: apis to support configurable " Honnappa Nagarahalli
2019-09-06 19:05   ` [dpdk-dev] [PATCH v2 2/6] lib/ring: add template to support different element sizes Honnappa Nagarahalli
2019-09-08 19:44     ` Stephen Hemminger
2019-09-09  9:01       ` Bruce Richardson
2019-09-09 22:33         ` Honnappa Nagarahalli
2019-09-06 19:05   ` [dpdk-dev] [PATCH v2 3/6] tools/checkpatch: relax constraints on __rte_experimental Honnappa Nagarahalli
2019-09-06 19:05   ` [dpdk-dev] [PATCH v2 4/6] lib/ring: add ring APIs to support 32b ring elements Honnappa Nagarahalli
2019-09-06 19:05   ` [dpdk-dev] [PATCH v2 5/6] lib/hash: use ring with 32b element size to save memory Honnappa Nagarahalli
2019-09-06 19:05   ` [dpdk-dev] [PATCH v2 6/6] lib/eventdev: use ring templates for event rings Honnappa Nagarahalli
2019-09-09 13:04   ` [dpdk-dev] [PATCH v2 0/6] lib/ring: templates to support custom element size Aaron Conole
2019-10-07 13:49   ` David Marchand
2019-10-08 19:19   ` [dpdk-dev] [PATCH v3 0/2] lib/ring: APIs " Honnappa Nagarahalli
2019-10-08 19:19     ` [dpdk-dev] [PATCH v3 1/2] lib/ring: apis to support configurable " Honnappa Nagarahalli
2019-10-08 19:19     ` [dpdk-dev] [PATCH v3 2/2] test/ring: add test cases for configurable element size ring Honnappa Nagarahalli
2019-10-09  2:47   ` [dpdk-dev] [PATCH v3 0/2] lib/ring: APIs to support custom element size Honnappa Nagarahalli
2019-10-09  2:47     ` [dpdk-dev] [PATCH v4 1/2] lib/ring: apis to support configurable " Honnappa Nagarahalli
2019-10-11 19:21       ` Honnappa Nagarahalli
2019-10-14 19:41         ` Ananyev, Konstantin
2019-10-14 23:56           ` Honnappa Nagarahalli
2019-10-15  9:34             ` Ananyev, Konstantin
2019-10-17  4:46               ` Honnappa Nagarahalli
2019-10-17 11:51                 ` Ananyev, Konstantin
2019-10-17 20:16                   ` Honnappa Nagarahalli
2019-10-17 23:17                     ` David Christensen
2019-10-18  3:18                       ` Honnappa Nagarahalli
2019-10-18  8:04                         ` Jerin Jacob
2019-10-18 16:11                           ` Jerin Jacob
2019-10-21  0:27                             ` Honnappa Nagarahalli
2019-10-18 16:44                           ` Ananyev, Konstantin
2019-10-18 19:03                             ` Honnappa Nagarahalli
2019-10-21  0:36                             ` Honnappa Nagarahalli
2019-10-21  9:04                               ` Ananyev, Konstantin
2019-10-22 15:59                                 ` Ananyev, Konstantin
2019-10-22 17:57                                   ` Ananyev, Konstantin
2019-10-23 18:58                                     ` Honnappa Nagarahalli
2019-10-18 17:23                         ` David Christensen
2019-10-09  2:47     ` [dpdk-dev] [PATCH v4 2/2] test/ring: add test cases for configurable element size ring Honnappa Nagarahalli
2019-10-17 20:08   ` [dpdk-dev] [PATCH v5 0/3] lib/ring: APIs to support custom element size Honnappa Nagarahalli
2019-10-17 20:08     ` [dpdk-dev] [PATCH v5 1/3] lib/ring: apis to support configurable " Honnappa Nagarahalli
2019-10-17 20:39       ` Stephen Hemminger
2019-10-17 20:40       ` Stephen Hemminger
2019-10-17 20:08     ` [dpdk-dev] [PATCH v5 2/3] test/ring: add test cases for configurable element size ring Honnappa Nagarahalli
2019-10-17 20:08     ` [dpdk-dev] [PATCH v5 3/3] lib/ring: copy ring elements using memcpy partially Honnappa Nagarahalli
2019-10-21  0:22   ` [dpdk-dev] [RFC v6 0/6] lib/ring: APIs to support custom element size Honnappa Nagarahalli
2019-10-21  0:22     ` [dpdk-dev] [RFC v6 1/6] test/ring: use division for cycle count calculation Honnappa Nagarahalli
2019-10-23  9:49       ` Olivier Matz
2019-10-21  0:22     ` [dpdk-dev] [RFC v6 2/6] lib/ring: apis to support configurable element size Honnappa Nagarahalli
2019-10-23  9:59       ` Olivier Matz
2019-10-23 19:12         ` Honnappa Nagarahalli
2019-10-21  0:22     ` [dpdk-dev] [RFC v6 3/6] test/ring: add functional tests for configurable element size ring Honnappa Nagarahalli
2019-10-23 10:01       ` Olivier Matz
2019-10-23 11:12         ` Ananyev, Konstantin
2019-10-21  0:22     ` [dpdk-dev] [RFC v6 4/6] test/ring: add perf " Honnappa Nagarahalli
2019-10-23 10:02       ` Olivier Matz
2019-10-21  0:22     ` [dpdk-dev] [RFC v6 5/6] lib/ring: copy ring elements using memcpy partially Honnappa Nagarahalli
2019-10-21  0:23     ` [dpdk-dev] [RFC v6 6/6] lib/ring: improved copy function to copy ring elements Honnappa Nagarahalli
2019-10-23 10:05       ` Olivier Matz
2019-10-23  9:48     ` [dpdk-dev] [RFC v6 0/6] lib/ring: APIs to support custom element size Olivier Matz
2019-12-20  4:45   ` [dpdk-dev] [PATCH v7 00/17] " Honnappa Nagarahalli
2019-12-20  4:45     ` [dpdk-dev] [PATCH v7 01/17] test/ring: use division for cycle count calculation Honnappa Nagarahalli
2019-12-20  4:45     ` [dpdk-dev] [PATCH v7 02/17] lib/ring: apis to support configurable element size Honnappa Nagarahalli
2020-01-02 16:42       ` Ananyev, Konstantin
2020-01-07  5:35         ` Honnappa Nagarahalli
2020-01-07  6:00           ` Honnappa Nagarahalli
2020-01-07 10:21             ` Ananyev, Konstantin
2020-01-07 15:21               ` Honnappa Nagarahalli
2020-01-07 15:41                 ` Ananyev, Konstantin
2020-01-08  6:17                   ` Honnappa Nagarahalli
2020-01-08 10:05                     ` Ananyev, Konstantin
2020-01-08 23:40                       ` Honnappa Nagarahalli
2020-01-09  0:48                         ` Ananyev, Konstantin
2020-01-09 16:06                           ` Honnappa Nagarahalli
2020-01-13 11:53                             ` Ananyev, Konstantin
2019-12-20  4:45     ` [dpdk-dev] [PATCH v7 03/17] test/ring: add functional tests for rte_ring_xxx_elem APIs Honnappa Nagarahalli
2020-01-02 16:31       ` Ananyev, Konstantin
2020-01-07  5:13         ` Honnappa Nagarahalli
2020-01-07 16:03           ` Ananyev, Konstantin
2020-01-09  5:15             ` Honnappa Nagarahalli
2019-12-20  4:45     ` [dpdk-dev] [PATCH v7 04/17] test/ring: test burst APIs with random empty-full test case Honnappa Nagarahalli
2019-12-20  4:45     ` [dpdk-dev] [PATCH v7 05/17] test/ring: add default, single element test cases Honnappa Nagarahalli
2019-12-20  4:45     ` [dpdk-dev] [PATCH v7 06/17] test/ring: rte_ring_xxx_elem test cases for exact size ring Honnappa Nagarahalli
2019-12-20  4:45     ` [dpdk-dev] [PATCH v7 07/17] test/ring: negative test cases for rte_ring_xxx_elem APIs Honnappa Nagarahalli
2019-12-20  4:45     ` [dpdk-dev] [PATCH v7 08/17] test/ring: remove duplicate test cases Honnappa Nagarahalli
2019-12-20  4:45     ` [dpdk-dev] [PATCH v7 09/17] test/ring: removed unused variable synchro Honnappa Nagarahalli
2019-12-20  4:45     ` [dpdk-dev] [PATCH v7 10/17] test/ring: modify single element enq/deq perf test cases Honnappa Nagarahalli
2020-01-02 17:03       ` Ananyev, Konstantin
2020-01-07  5:54         ` Honnappa Nagarahalli
2020-01-07 16:13           ` Ananyev, Konstantin
2020-01-07 22:33             ` Honnappa Nagarahalli
2019-12-20  4:45     ` [dpdk-dev] [PATCH v7 11/17] test/ring: modify burst " Honnappa Nagarahalli
2020-01-02 16:57       ` Ananyev, Konstantin
2020-01-07  5:42         ` Honnappa Nagarahalli
2019-12-20  4:45     ` [dpdk-dev] [PATCH v7 12/17] test/ring: modify bulk " Honnappa Nagarahalli
2019-12-20  4:45     ` [dpdk-dev] [PATCH v7 13/17] test/ring: modify bulk empty deq " Honnappa Nagarahalli
2019-12-20  4:45     ` [dpdk-dev] [PATCH v7 14/17] test/ring: modify multi-lcore " Honnappa Nagarahalli
2019-12-20  4:45     ` [dpdk-dev] [PATCH v7 15/17] test/ring: adjust run-on-all-cores " Honnappa Nagarahalli
2020-01-02 17:00       ` Ananyev, Konstantin
2020-01-07  5:42         ` Honnappa Nagarahalli
2019-12-20  4:45     ` [dpdk-dev] [PATCH v7 16/17] lib/hash: use ring with 32b element size to save memory Honnappa Nagarahalli
2019-12-20  4:45     ` [dpdk-dev] [PATCH v7 17/17] lib/eventdev: use custom element size ring for event rings Honnappa Nagarahalli
2020-01-13 17:25   ` [dpdk-dev] [PATCH v8 0/6] lib/ring: APIs to support custom element size Honnappa Nagarahalli
2020-01-13 17:25     ` [dpdk-dev] [PATCH v8 1/6] test/ring: use division for cycle count calculation Honnappa Nagarahalli
2020-01-13 17:25     ` [dpdk-dev] [PATCH v8 2/6] lib/ring: apis to support configurable element size Honnappa Nagarahalli
2020-01-13 17:25     ` [dpdk-dev] [PATCH v8 3/6] test/ring: add functional tests for rte_ring_xxx_elem APIs Honnappa Nagarahalli
2020-01-13 17:25     ` [dpdk-dev] [PATCH v8 4/6] test/ring: modify perf test cases to use " Honnappa Nagarahalli
2020-01-13 17:25     ` [dpdk-dev] [PATCH v8 5/6] lib/hash: use ring with 32b element size to save memory Honnappa Nagarahalli
2020-01-13 17:25     ` [dpdk-dev] [PATCH v8 6/6] lib/eventdev: use custom element size ring for event rings Honnappa Nagarahalli
     [not found]       ` <1578977880-13011-1-git-send-email-robot@bytheb.org>
     [not found]         ` <VE1PR08MB5149BE79083CD66A41CBD6D198340@VE1PR08MB5149.eurprd08.prod.outlook.com>
2020-01-14 15:12           ` [dpdk-dev] FW: || pw64572 " Aaron Conole
2020-01-14 16:51             ` Aaron Conole
2020-01-14 19:35               ` Honnappa Nagarahalli
2020-01-14 20:44                 ` Aaron Conole
2020-01-15  0:55                   ` Honnappa Nagarahalli
2020-01-15  4:43                   ` Honnappa Nagarahalli
2020-01-15  5:05                     ` Honnappa Nagarahalli
2020-01-15 18:22                       ` Aaron Conole
2020-01-15 18:38                         ` Honnappa Nagarahalli
2020-01-16  5:27                           ` Honnappa Nagarahalli
2020-01-16  5:25   ` [dpdk-dev] [PATCH v9 0/6] lib/ring: APIs to support custom element size Honnappa Nagarahalli
2020-01-16  5:25     ` [dpdk-dev] [PATCH v9 1/6] test/ring: use division for cycle count calculation Honnappa Nagarahalli
2020-01-16  5:25     ` [dpdk-dev] [PATCH v9 2/6] lib/ring: apis to support configurable element size Honnappa Nagarahalli
2020-01-17 16:34       ` Olivier Matz
2020-01-17 16:45         ` Honnappa Nagarahalli
2020-01-17 18:10           ` David Christensen
2020-01-18 12:32           ` Ananyev, Konstantin
2020-01-18 15:01             ` Honnappa Nagarahalli
2020-01-16  5:25     ` [dpdk-dev] [PATCH v9 3/6] test/ring: add functional tests for rte_ring_xxx_elem APIs Honnappa Nagarahalli
2020-01-17 17:03       ` Olivier Matz
2020-01-18 16:27         ` Honnappa Nagarahalli
2020-01-16  5:25     ` [dpdk-dev] [PATCH v9 4/6] test/ring: modify perf test cases to use " Honnappa Nagarahalli
2020-01-17 17:12       ` Olivier Matz
2020-01-18 16:28         ` Honnappa Nagarahalli
2020-01-16  5:25     ` [dpdk-dev] [PATCH v9 5/6] lib/hash: use ring with 32b element size to save memory Honnappa Nagarahalli
2020-01-17 20:27       ` David Marchand
2020-01-17 20:54         ` Honnappa Nagarahalli
2020-01-17 21:07           ` David Marchand
2020-01-17 22:24             ` Wang, Yipeng1
2020-01-16  5:25     ` [dpdk-dev] [PATCH v9 6/6] lib/eventdev: use custom element size ring for event rings Honnappa Nagarahalli
2020-01-17 14:41       ` Jerin Jacob
2020-01-17 16:12         ` David Marchand
2020-01-16 16:36     ` [dpdk-dev] [PATCH v9 0/6] lib/ring: APIs to support custom element size Honnappa Nagarahalli
2020-01-17 12:14       ` David Marchand
2020-01-17 13:34         ` Jerin Jacob
2020-01-17 16:37           ` Mattias Rönnblom
2020-01-17 14:28         ` Honnappa Nagarahalli
2020-01-17 14:36           ` Honnappa Nagarahalli
2020-01-17 16:15           ` David Marchand
2020-01-17 16:32             ` Honnappa Nagarahalli
2020-01-17 17:15     ` Olivier Matz
2020-01-18 19:32   ` Honnappa Nagarahalli [this message]
2020-01-18 19:32     ` [dpdk-dev] [PATCH v10 1/6] test/ring: use division for cycle count calculation Honnappa Nagarahalli
2020-01-18 19:32     ` [dpdk-dev] [PATCH v10 2/6] lib/ring: apis to support configurable element size Honnappa Nagarahalli
2020-01-18 19:32     ` [dpdk-dev] [PATCH v10 3/6] test/ring: add functional tests for rte_ring_xxx_elem APIs Honnappa Nagarahalli
2020-01-18 19:32     ` [dpdk-dev] [PATCH v10 4/6] test/ring: modify perf test cases to use " Honnappa Nagarahalli
2020-01-18 19:32     ` [dpdk-dev] [PATCH v10 5/6] lib/hash: use ring with 32b element size to save memory Honnappa Nagarahalli
2020-01-18 19:32     ` [dpdk-dev] [PATCH v10 6/6] eventdev: use custom element size ring for event rings Honnappa Nagarahalli
2020-01-19 19:31     ` [dpdk-dev] [PATCH v10 0/6] lib/ring: APIs to support custom element size David Marchand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200118193247.43831-1-honnappa.nagarahalli@arm.com \
    --to=honnappa.nagarahalli@arm.com \
    --cc=bruce.richardson@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=dharmik.thakkar@arm.com \
    --cc=drc@linux.vnet.ibm.com \
    --cc=gavin.hu@arm.com \
    --cc=jerinj@marvell.com \
    --cc=konstantin.ananyev@intel.com \
    --cc=nd@arm.com \
    --cc=olivier.matz@6wind.com \
    --cc=pbhagavatula@marvell.com \
    --cc=ruifeng.wang@arm.com \
    --cc=sthemmin@microsoft.com \
    --cc=yipeng1.wang@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).