DPDK patches and discussions
 help / color / mirror / Atom feed
From: Ruifeng Wang <ruifeng.wang@arm.com>
To: bruce.richardson@intel.com, vladimir.medvedkin@intel.com,
	olivier.matz@6wind.com
Cc: dev@dpdk.org, honnappa.nagarahalli@arm.com,
	dharmik.thakkar@arm.com, nd@arm.com
Subject: [dpdk-dev] [RFC PATCH 1/3] doc/rcu: add RCU integration design details
Date: Thu, 22 Aug 2019 14:34:55 +0800
Message-ID: <20190822063457.41596-2-ruifeng.wang@arm.com> (raw)
In-Reply-To: <20190822063457.41596-1-ruifeng.wang@arm.com>

From: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>

Add a section to describe a design to integrate QSBR RCU library
with other libraries in DPDK.

Signed-off-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
---
 doc/guides/prog_guide/rcu_lib.rst | 51 +++++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)

diff --git a/doc/guides/prog_guide/rcu_lib.rst b/doc/guides/prog_guide/rcu_lib.rst
index 8fe5b1f73..2869441ca 100644
--- a/doc/guides/prog_guide/rcu_lib.rst
+++ b/doc/guides/prog_guide/rcu_lib.rst
@@ -186,3 +186,54 @@ However, when ``CONFIG_RTE_LIBRTE_RCU_DEBUG`` is enabled, these APIs aid
 in debugging issues. One can mark the access to shared data structures on the
 reader side using these APIs. The ``rte_rcu_qsbr_quiescent()`` will check if
 all the locks are unlocked.
+
+Integrating QSBR RCU with other libraries
+-----------------------------------------
+
+Lock-free algorithms place additional burden on the application to reclaim
+memory. Integrating memory reclaiming mechanisms in the libraries help
+remove some of the burden. Though QSBR method presents flexibility to
+achieve performance, it presents challenges while integrating with libraries.
+
+The memory reclaiming process using QSBR can be split into 4 parts:
+
+#. Initialization
+#. Quiescent State Reporting
+#. Reclaiming Resources
+#. Shutdown
+
+The design proposed here requires the application to handle 'Initialization'
+and 'Quiescent State Reporting'. So,
+
+* the application has to create the RCU variable and register the reader threads to report their quiescent state.
+* the application has to register the same RCU variable with the library.
+* reader threads in the application have to report the quiescent state. This allows for the application to control the length of the critical section/how frequently the application wants to report the quiescent state.
+
+The library will handle 'Reclaiming Resources' part of the process. The
+libraries will make use of the writer thread context to execute the memory
+reclaiming algorithm. So,
+
+* library should provide an API to register a RCU variable that it will use.
+* library should trigger the readers to report quiescent state status upon deleting the resources by calling ``rte_rcu_qsbr_start``.
+
+* library should store the token and deleted resources for later use to free them after the readers have reported their quiescent state. Since the readers will report the quiescent state status in the order of deletion, the library must store the tokens/resources in the order in which the resources were deleted. A FIFO data structure would achieve the desired results. The length of the FIFO would depend on the rate of deletion and the rate at which the readers report their quiescent state. In the worst case the length of FIFO would be equal to the maximum number of resources the data structure supports. However, in most cases, the length will be much smaller. But, the library should not take the length of FIFO as an input from the application. Instead, it should implement a data structure which should be able to grow/shrink dynamically. Overhead introduced by such a data structure on delete operations should be considered as well.
+
+* library should query the quiescent state and free the resources. It should make use of non-blocking ``rte_rcu_qsbr_check`` API to query the quiescent state. This allows the application to do useful work while the readers report their quiescent state. If there are tokens/resources present in the FIFO already, the delete API should peek the head of the FIFO and check the quiescent state status. If the status is success, the token/resource should be dequeued and the resource should be freed. This process can be repeated till the quiescent state status for a token returns failure indicating that subsequent tokens will also fail quiescent state status query. The same process can be incorporated while adding new entries in the data structure if the library runs out of resources.
+
+The 'Shutdown' process needs to be shared between the application and the
+library.
+
+* library should check the quiescent state status of all the tokens that may be present in the FIFO and free the resources. It should make use of non-blocking ``rte_rcu_qsbr_check`` API to query the quiescent state. If any of the tokens do not pass the quiescent state check, the library should print an error and stop the memory reclaimation process.
+
+* the application should make sure that the reader threads are not using the shared data structure, unregister the reader threads from the QSBR variable before calling the library's shutdown function.
+
+Integrating the resource reclaimation with libraries removes the burden from
+the application and makes it easy to use lock-free algorithms.
+
+This design has several advantages over currently known methods.
+
+#. Application does not need a dedicated thread to reclaim resources. Memory
+   reclaimation happens as part of the writer thread without sacrificing
+   a lot of performance.
+#. The library has better control over the resources. For ex: the library can
+   attempt to reclaim when it has run out of resources.
-- 
2.17.1


  reply	other threads:[~2019-08-22  6:35 UTC|newest]

Thread overview: 137+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-22  6:34 [dpdk-dev] [RFC PATCH 0/3] RCU integration with LPM library Ruifeng Wang
2019-08-22  6:34 ` Ruifeng Wang [this message]
2019-08-22  6:34 ` [dpdk-dev] [RFC PATCH 2/3] lib/ring: add peek API Ruifeng Wang
2019-08-22  6:34 ` [dpdk-dev] [RFC PATCH 3/3] lib/lpm: integrate RCU QSBR Ruifeng Wang
2019-08-23  1:23   ` Stephen Hemminger
2019-08-26  3:11     ` Ruifeng Wang (Arm Technology China)
2019-08-26  5:32       ` Honnappa Nagarahalli
2019-08-22 15:52 ` [dpdk-dev] [RFC PATCH 0/3] RCU integration with LPM library Honnappa Nagarahalli
2019-09-06  9:45 ` [dpdk-dev] [PATCH v2 0/6] " Ruifeng Wang
2019-09-06  9:45   ` [dpdk-dev] [PATCH v2 1/6] doc/rcu: add RCU integration design details Ruifeng Wang
2019-09-06 19:44     ` Honnappa Nagarahalli
2019-09-06  9:45   ` [dpdk-dev] [PATCH v2 2/6] lib/ring: add peek API Ruifeng Wang
2019-09-06  9:45   ` [dpdk-dev] [PATCH v2 3/6] lib/lpm: integrate RCU QSBR Ruifeng Wang
2019-09-06 19:44     ` Honnappa Nagarahalli
2019-09-18 16:15     ` Medvedkin, Vladimir
2019-09-19  6:17       ` Ruifeng Wang (Arm Technology China)
2019-09-06  9:45   ` [dpdk-dev] [PATCH v2 4/6] app/test: add test case for LPM RCU integration Ruifeng Wang
2019-09-06 19:45     ` Honnappa Nagarahalli
2019-09-06  9:45   ` [dpdk-dev] [PATCH v2 5/6] test/lpm: reset total time Ruifeng Wang
2019-09-18 16:17     ` Medvedkin, Vladimir
2019-09-19  6:22       ` Ruifeng Wang (Arm Technology China)
2019-09-06  9:45   ` [dpdk-dev] [PATCH v2 6/6] test/lpm: add RCU integration performance tests Ruifeng Wang
2019-09-06 19:46     ` Honnappa Nagarahalli
2019-10-01  6:29   ` [dpdk-dev] [PATCH v3 0/3] Add RCU reclamation APIs Honnappa Nagarahalli
2019-10-01  6:29     ` [dpdk-dev] [PATCH v3 1/3] lib/ring: add peek API Honnappa Nagarahalli
2019-10-02 18:42       ` Ananyev, Konstantin
2019-10-03 19:49         ` Honnappa Nagarahalli
2019-10-07  9:01           ` Ananyev, Konstantin
2019-10-09  4:25             ` Honnappa Nagarahalli
2019-10-10 15:09               ` Ananyev, Konstantin
2019-10-11  5:03                 ` Honnappa Nagarahalli
2019-10-11 14:41                   ` Ananyev, Konstantin
2019-10-11 18:28                     ` Honnappa Nagarahalli
2019-10-13 20:09                       ` Ananyev, Konstantin
2019-10-14  4:11                         ` Honnappa Nagarahalli
2019-10-01  6:29     ` [dpdk-dev] [PATCH v3 2/3] lib/rcu: add resource reclamation APIs Honnappa Nagarahalli
2019-10-02 17:39       ` Ananyev, Konstantin
2019-10-03  6:29         ` Honnappa Nagarahalli
2019-10-03 12:26           ` Ananyev, Konstantin
2019-10-04  6:07             ` Honnappa Nagarahalli
2019-10-07 10:46               ` Ananyev, Konstantin
2019-10-13  4:35                 ` Honnappa Nagarahalli
2019-10-02 18:50       ` Ananyev, Konstantin
2019-10-03  6:42         ` Honnappa Nagarahalli
2019-10-03 11:52           ` Ananyev, Konstantin
2019-10-04 19:01       ` Medvedkin, Vladimir
2019-10-07 13:11       ` Medvedkin, Vladimir
2019-10-13  3:02         ` Honnappa Nagarahalli
2019-10-15 16:48           ` Medvedkin, Vladimir
2019-10-18  3:47             ` Honnappa Nagarahalli
2019-10-01  6:29     ` [dpdk-dev] [PATCH v3 3/3] doc/rcu: add RCU integration design details Honnappa Nagarahalli
2020-03-29 20:57     ` [dpdk-dev] [PATCH v3 0/3] Add RCU reclamation APIs Thomas Monjalon
2020-03-30 17:37       ` Honnappa Nagarahalli
2020-04-03 18:41     ` [dpdk-dev] [PATCH v4 0/4] " Honnappa Nagarahalli
2020-04-03 18:41       ` [dpdk-dev] [PATCH v4 1/4] lib/rcu: add resource " Honnappa Nagarahalli
2020-04-07 17:39         ` Ananyev, Konstantin
2020-04-19 23:22           ` Honnappa Nagarahalli
2020-04-20  8:19             ` Ananyev, Konstantin
2020-04-03 18:41       ` [dpdk-dev] [PATCH v4 2/4] test/rcu: test cases for RCU defer queue APIs Honnappa Nagarahalli
2020-04-03 18:41       ` [dpdk-dev] [PATCH v4 3/4] doc/rcu: add RCU integration design details Honnappa Nagarahalli
2020-04-03 18:41       ` [dpdk-dev] [PATCH v4 4/4] lib/rcu: add additional debug logs Honnappa Nagarahalli
2020-04-22  3:30     ` [dpdk-dev] [PATCH v5 0/4] Add RCU reclamation APIs Honnappa Nagarahalli
2020-04-22  3:30       ` [dpdk-dev] [PATCH v5 1/4] lib/rcu: add resource " Honnappa Nagarahalli
2020-04-22  8:36         ` Ananyev, Konstantin
2020-04-22  8:42           ` David Marchand
2020-04-22  8:51             ` David Marchand
2020-04-22  9:26               ` Ananyev, Konstantin
2020-04-22  3:30       ` [dpdk-dev] [PATCH v5 2/4] test/rcu: test cases for RCU defer queue APIs Honnappa Nagarahalli
2020-04-22  8:27         ` Ananyev, Konstantin
2020-04-22  3:30       ` [dpdk-dev] [PATCH v5 3/4] doc/rcu: add RCU integration design details Honnappa Nagarahalli
2020-04-22  3:30       ` [dpdk-dev] [PATCH v5 4/4] lib/rcu: add additional debug logs Honnappa Nagarahalli
2020-04-22  8:25         ` Ananyev, Konstantin
2020-04-22 18:46       ` [dpdk-dev] [PATCH v5 0/4] Add RCU reclamation APIs David Marchand
2019-10-01 18:28   ` [dpdk-dev] [PATCH v3 0/3] RCU integration with LPM library Honnappa Nagarahalli
2019-10-01 18:28     ` [dpdk-dev] [PATCH v3 1/3] lib/lpm: integrate RCU QSBR Honnappa Nagarahalli
2019-10-04 16:05       ` Medvedkin, Vladimir
2019-10-09  3:48         ` Honnappa Nagarahalli
2019-10-07  9:21       ` Ananyev, Konstantin
2019-10-13  4:36         ` Honnappa Nagarahalli
2019-10-15 11:15           ` Ananyev, Konstantin
2019-10-18  3:32             ` Honnappa Nagarahalli
2019-10-01 18:28     ` [dpdk-dev] [PATCH v3 2/3] app/test: add test case for LPM RCU integration Honnappa Nagarahalli
2019-10-01 18:28     ` [dpdk-dev] [PATCH v3 3/3] test/lpm: add RCU integration performance tests Honnappa Nagarahalli
2019-10-02 13:02       ` Aaron Conole
2019-10-03  9:09         ` Bruce Richardson
2020-06-08  5:16   ` [dpdk-dev] [PATCH v4 0/3] RCU integration with LPM library Ruifeng Wang
2020-06-08  5:16     ` [dpdk-dev] [PATCH v4 1/3] lib/lpm: integrate RCU QSBR Ruifeng Wang
2020-06-08 18:46       ` Honnappa Nagarahalli
2020-06-18 17:36         ` Medvedkin, Vladimir
2020-06-18 17:21       ` Medvedkin, Vladimir
2020-06-22  5:46         ` Ruifeng Wang
2020-06-23  4:34           ` Honnappa Nagarahalli
2020-06-08  5:16     ` [dpdk-dev] [PATCH v4 2/3] test/lpm: add LPM RCU integration functional tests Ruifeng Wang
2020-06-08  5:16     ` [dpdk-dev] [PATCH v4 3/3] test/lpm: add RCU integration performance tests Ruifeng Wang
2020-06-29  8:02   ` [dpdk-dev] [PATCH v5 0/3] RCU integration with LPM library Ruifeng Wang
2020-06-29  8:02     ` [dpdk-dev] [PATCH v5 1/3] lib/lpm: integrate RCU QSBR Ruifeng Wang
2020-06-29 11:56       ` David Marchand
2020-06-29 12:55         ` Bruce Richardson
2020-06-30 10:35           ` Kinsella, Ray
2020-07-03  7:43         ` David Marchand
2020-07-04 17:00         ` Ruifeng Wang
2020-06-30 10:33       ` Kinsella, Ray
2020-06-29  8:02     ` [dpdk-dev] [PATCH v5 2/3] test/lpm: add LPM RCU integration functional tests Ruifeng Wang
2020-06-29  8:03     ` [dpdk-dev] [PATCH v5 3/3] test/lpm: add RCU integration performance tests Ruifeng Wang
2020-07-07 14:40   ` [dpdk-dev] [PATCH v6 0/3] RCU integration with LPM library Ruifeng Wang
2020-07-07 14:40     ` [dpdk-dev] [PATCH v6 1/3] lib/lpm: integrate RCU QSBR Ruifeng Wang
2020-07-07 14:40     ` [dpdk-dev] [PATCH v6 2/3] test/lpm: add LPM RCU integration functional tests Ruifeng Wang
2020-07-07 14:40     ` [dpdk-dev] [PATCH v6 3/3] test/lpm: add RCU integration performance tests Ruifeng Wang
2020-07-07 15:15   ` [dpdk-dev] [PATCH v7 0/3] RCU integration with LPM library Ruifeng Wang
2020-07-07 15:15     ` [dpdk-dev] [PATCH v7 1/3] lib/lpm: integrate RCU QSBR Ruifeng Wang
2020-07-08 12:36       ` Medvedkin, Vladimir
2020-07-08 14:30       ` David Marchand
2020-07-08 15:34         ` Ruifeng Wang
2020-07-07 15:15     ` [dpdk-dev] [PATCH v7 2/3] test/lpm: add LPM RCU integration functional tests Ruifeng Wang
2020-07-08 12:37       ` Medvedkin, Vladimir
2020-07-08 14:00         ` Ruifeng Wang
2020-07-07 15:15     ` [dpdk-dev] [PATCH v7 3/3] test/lpm: add RCU integration performance tests Ruifeng Wang
2020-07-08 12:37       ` Medvedkin, Vladimir
2020-07-08 14:07         ` Ruifeng Wang
2020-07-09  8:02   ` [dpdk-dev] [PATCH v8 0/3] RCU integration with LPM library Ruifeng Wang
2020-07-09  8:02     ` [dpdk-dev] [PATCH v8 1/3] lib/lpm: integrate RCU QSBR Ruifeng Wang
2020-07-09 11:49       ` David Marchand
2020-07-09 14:35         ` Ruifeng Wang
2020-07-09  8:02     ` [dpdk-dev] [PATCH v8 2/3] test/lpm: add LPM RCU integration functional tests Ruifeng Wang
2020-07-09  8:02     ` [dpdk-dev] [PATCH v8 3/3] test/lpm: add RCU integration performance tests Ruifeng Wang
2020-07-09 15:42   ` [dpdk-dev] [PATCH v9 0/3] RCU integration with LPM library Ruifeng Wang
2020-07-09 15:42     ` [dpdk-dev] [PATCH v9 1/3] lib/lpm: integrate RCU QSBR Ruifeng Wang
2020-07-09 15:42     ` [dpdk-dev] [PATCH v9 2/3] test/lpm: add LPM RCU integration functional tests Ruifeng Wang
2020-07-09 15:42     ` [dpdk-dev] [PATCH v9 3/3] test/lpm: add RCU integration performance tests Ruifeng Wang
2020-07-10  2:22   ` [dpdk-dev] [PATCH v10 0/3] RCU integration with LPM library Ruifeng Wang
2020-07-10  2:22     ` [dpdk-dev] [PATCH v10 1/3] lib/lpm: integrate RCU QSBR Ruifeng Wang
2020-07-10  2:29       ` Ruifeng Wang
2020-07-10  2:22     ` [dpdk-dev] [PATCH v10 2/3] test/lpm: add LPM RCU integration functional tests Ruifeng Wang
2020-07-10  2:22     ` [dpdk-dev] [PATCH v10 3/3] test/lpm: add RCU integration performance tests Ruifeng Wang
2020-07-10  2:29       ` Ruifeng Wang
2020-07-10 12:21     ` [dpdk-dev] [PATCH v10 0/3] RCU integration with LPM library David Marchand
2020-07-10 14:34       ` Ruifeng Wang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190822063457.41596-2-ruifeng.wang@arm.com \
    --to=ruifeng.wang@arm.com \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=dharmik.thakkar@arm.com \
    --cc=honnappa.nagarahalli@arm.com \
    --cc=nd@arm.com \
    --cc=olivier.matz@6wind.com \
    --cc=vladimir.medvedkin@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git