From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A6B2AA0350; Tue, 30 Jun 2020 12:36:03 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8AEAA1BEB3; Tue, 30 Jun 2020 12:36:03 +0200 (CEST) Received: from dal2relay51.mxroute.com (dal2relay51.mxroute.com [64.40.26.51]) by dpdk.org (Postfix) with ESMTP id ED1271BE8E for ; Tue, 30 Jun 2020 12:36:01 +0200 (CEST) Received: from filter004.mxroute.com ([149.28.56.236] 149.28.56.236.vultr.com) (Authenticated sender: mN4UYu2MZsgR) by dal2relay51.mxroute.com (ZoneMTA) with ESMTPSA id 17304cc2e280001663.004 for (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES128-GCM-SHA256); Tue, 30 Jun 2020 10:35:56 +0000 X-Zone-Loop: ceb67d8d2ba2d474452f275aa3e2e0a84e23689d2151 X-Originating-IP: [149.28.56.236] Received: from echo.mxrouting.net (echo.mxrouting.net [116.202.222.109]) by filter004.mxroute.com (Postfix) with ESMTPS id E06833E9F0; Tue, 30 Jun 2020 10:35:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ashroe.eu; s=x; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version:Date: Message-ID:From:References:Cc:To:Subject:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=4deALGA6eI1g92Ceq2/S5q7E9RWoOPrNKY4VH+PVDt4=; b=XvVhi9RgY+bVf6BnBfYu17AQ1J o4vhgvh+nqOS4Zg6pLMlwalugHvjW5Hvc4dfnV7N0qGPK2hUwHkyGRgx2qBrvfRs9zJ3cEU4JGZgI AuuWqCh4x+9rLV9f0JHmOHd7au72DdWzfqNgcSr9HMxCestQc2iWfLYYD0m7NHV5xoD9jW6AWSJIo enUFmBcT4b7vJr04RtW/cN6rTgScHcrVvUQD7GhTplTMXvbSuEh/XQJYj8n3wwEByeGHzRT+qRODR OYQSo/oLLmdbrjiZk8itRGIJNwlOnGn2Kq3c8/4eJoP/NUl4zshAEAKMMhSxwioZMVtAENRUi+mHF IUVLD/Hg==; To: Bruce Richardson , David Marchand Cc: Ruifeng Wang , Vladimir Medvedkin , John McNamara , Marko Kovacevic , Neil Horman , dev , "Ananyev, Konstantin" , Honnappa Nagarahalli , nd References: <20190906094534.36060-1-ruifeng.wang@arm.com> <20200629080301.97515-1-ruifeng.wang@arm.com> <20200629080301.97515-2-ruifeng.wang@arm.com> <20200629125514.GC572@bricha3-MOBL.ger.corp.intel.com> From: "Kinsella, Ray" Autocrypt: addr=mdr@ashroe.eu; keydata= mQINBFv8B3wBEAC+5ImcgbIvadt3axrTnt7Sxch3FsmWTTomXfB8YiuHT8KL8L/bFRQSL1f6 ASCHu3M89EjYazlY+vJUWLr0BhK5t/YI7bQzrOuYrl9K94vlLwzD19s/zB/g5YGGR5plJr0s JtJsFGEvF9LL3e+FKMRXveQxBB8A51nAHfwG0WSyx53d61DYz7lp4/Y4RagxaJoHp9lakn8j HV2N6rrnF+qt5ukj5SbbKWSzGg5HQF2t0QQ5tzWhCAKTfcPlnP0GymTBfNMGOReWivi3Qqzr S51Xo7hoGujUgNAM41sxpxmhx8xSwcQ5WzmxgAhJ/StNV9cb3HWIoE5StCwQ4uXOLplZNGnS uxNdegvKB95NHZjRVRChg/uMTGpg9PqYbTIFoPXjuk27sxZLRJRrueg4tLbb3HM39CJwSB++ YICcqf2N+GVD48STfcIlpp12/HI+EcDSThzfWFhaHDC0hyirHxJyHXjnZ8bUexI/5zATn/ux TpMbc/vicJxeN+qfaVqPkCbkS71cHKuPluM3jE8aNCIBNQY1/j87k5ELzg3qaesLo2n1krBH bKvFfAmQuUuJT84/IqfdVtrSCTabvDuNBDpYBV0dGbTwaRfE7i+LiJJclUr8lOvHUpJ4Y6a5 0cxEPxm498G12Z3NoY/mP5soItPIPtLR0rA0fage44zSPwp6cQARAQABtBxSYXkgS2luc2Vs bGEgPG1kckBhc2hyb2UuZXU+iQJUBBMBCAA+FiEEcDUDlKDJaDuJlfZfdJdaH/sCCpsFAlv8 B3wCGyMFCQlmAYAFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQdJdaH/sCCptdtRAAl0oE msa+djBVYLIsax+0f8acidtWg2l9f7kc2hEjp9h9aZCpPchQvhhemtew/nKavik3RSnLTAyn B3C/0GNlmvI1l5PFROOgPZwz4xhJKGN7jOsRrbkJa23a8ly5UXwF3Vqnlny7D3z+7cu1qq/f VRK8qFyWkAb+xgqeZ/hTcbJUWtW+l5Zb+68WGEp8hB7TuJLEWb4+VKgHTpQ4vElYj8H3Z94a 04s2PJMbLIZSgmKDASnyrKY0CzTpPXx5rSJ1q+B1FCsfepHLqt3vKSALa3ld6bJ8fSJtDUJ7 JLiU8dFZrywgDIVme01jPbjJtUScW6jONLvhI8Z2sheR71UoKqGomMHNQpZ03ViVWBEALzEt TcjWgJFn8yAmxqM4nBnZ+hE3LbMo34KCHJD4eg18ojDt3s9VrDLa+V9fNxUHPSib9FD9UX/1 +nGfU/ZABmiTuUDM7WZdXri7HaMpzDRJUKI6b+/uunF8xH/h/MHW16VuMzgI5dkOKKv1LejD dT5mA4R+2zBS+GsM0oa2hUeX9E5WwjaDzXtVDg6kYq8YvEd+m0z3M4e6diFeLS77/sAOgaYL 92UcoKD+Beym/fVuC6/55a0e12ksTmgk5/ZoEdoNQLlVgd2INtvnO+0k5BJcn66ZjKn3GbEC VqFbrnv1GnA58nEInRCTzR1k26h9nmS5Ag0EW/wHfAEQAMth1vHr3fOZkVOPfod3M6DkQir5 xJvUW5EHgYUjYCPIa2qzgIVVuLDqZgSCCinyooG5dUJONVHj3nCbITCpJp4eB3PI84RPfDcC hf/V34N/Gx5mTeoymSZDBmXT8YtvV/uJvn+LvHLO4ZJdvq5ZxmDyxfXFmkm3/lLw0+rrNdK5 pt6OnVlCqEU9tcDBezjUwDtOahyV20XqxtUttN4kQWbDRkhT+HrA9WN9l2HX91yEYC+zmF1S OhBqRoTPLrR6g4sCWgFywqztpvZWhyIicJipnjac7qL/wRS+wrWfsYy6qWLIV80beN7yoa6v ccnuy4pu2uiuhk9/edtlmFE4dNdoRf7843CV9k1yRASTlmPkU59n0TJbw+okTa9fbbQgbIb1 pWsAuicRHyLUIUz4f6kPgdgty2FgTKuPuIzJd1s8s6p2aC1qo+Obm2gnBTduB+/n1Jw+vKpt 07d+CKEKu4CWwvZZ8ktJJLeofi4hMupTYiq+oMzqH+V1k6QgNm0Da489gXllU+3EFC6W1qKj tkvQzg2rYoWeYD1Qn8iXcO4Fpk6wzylclvatBMddVlQ6qrYeTmSbCsk+m2KVrz5vIyja0o5Y yfeN29s9emXnikmNfv/dA5fpi8XCANNnz3zOfA93DOB9DBf0TQ2/OrSPGjB3op7RCfoPBZ7u AjJ9dM7VABEBAAGJAjwEGAEIACYWIQRwNQOUoMloO4mV9l90l1of+wIKmwUCW/wHfAIbDAUJ CWYBgAAKCRB0l1of+wIKm3KlD/9w/LOG5rtgtCUWPl4B3pZvGpNym6XdK8cop9saOnE85zWf u+sKWCrxNgYkYP7aZrYMPwqDvilxhbTsIJl5HhPgpTO1b0i+c0n1Tij3EElj5UCg3q8mEc17 c+5jRrY3oz77g7E3oPftAjaq1ybbXjY4K32o3JHFR6I8wX3m9wJZJe1+Y+UVrrjY65gZFxcA thNVnWKErarVQGjeNgHV4N1uF3pIx3kT1N4GSnxhoz4Bki91kvkbBhUgYfNflGURfZT3wIKK +d50jd7kqRouXUCzTdzmDh7jnYrcEFM4nvyaYu0JjSS5R672d9SK5LVIfWmoUGzqD4AVmUW8 pcv461+PXchuS8+zpltR9zajl72Q3ymlT4BTAQOlCWkD0snBoKNUB5d2EXPNV13nA0qlm4U2 GpROfJMQXjV6fyYRvttKYfM5xYKgRgtP0z5lTAbsjg9WFKq0Fndh7kUlmHjuAIwKIV4Tzo75 QO2zC0/NTaTjmrtiXhP+vkC4pcrOGNsbHuaqvsc/ZZ0siXyYsqbctj/sCd8ka2r94u+c7o4l BGaAm+FtwAfEAkXHu4y5Phuv2IRR+x1wTey1U1RaEPgN8xq0LQ1OitX4t2mQwjdPihZQBCnZ wzOrkbzlJMNrMKJpEgulmxAHmYJKgvZHXZXtLJSejFjR0GdHJcL5rwVOMWB8cg== Message-ID: <0d1d3cd3-fdda-6f6f-125b-27d170683cdb@ashroe.eu> Date: Tue, 30 Jun 2020 11:35:53 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 MIME-Version: 1.0 In-Reply-To: <20200629125514.GC572@bricha3-MOBL.ger.corp.intel.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-AuthUser: mdr@ashroe.eu Subject: Re: [dpdk-dev] [PATCH v5 1/3] lib/lpm: integrate RCU QSBR X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 29/06/2020 13:55, Bruce Richardson wrote: > On Mon, Jun 29, 2020 at 01:56:07PM +0200, David Marchand wrote: >> On Mon, Jun 29, 2020 at 10:03 AM Ruifeng Wang wrote: >>> >>> Currently, the tbl8 group is freed even though the readers might be >>> using the tbl8 group entries. The freed tbl8 group can be reallocated >>> quickly. This results in incorrect lookup results. >>> >>> RCU QSBR process is integrated for safe tbl8 group reclaim. >>> Refer to RCU documentation to understand various aspects of >>> integrating RCU library into other libraries. >>> >>> Signed-off-by: Ruifeng Wang >>> Reviewed-by: Honnappa Nagarahalli >>> --- >>> doc/guides/prog_guide/lpm_lib.rst | 32 +++++++ >>> lib/librte_lpm/Makefile | 2 +- >>> lib/librte_lpm/meson.build | 1 + >>> lib/librte_lpm/rte_lpm.c | 129 ++++++++++++++++++++++++++--- >>> lib/librte_lpm/rte_lpm.h | 59 +++++++++++++ >>> lib/librte_lpm/rte_lpm_version.map | 6 ++ >>> 6 files changed, 216 insertions(+), 13 deletions(-) >>> >>> diff --git a/doc/guides/prog_guide/lpm_lib.rst b/doc/guides/prog_guide/lpm_lib.rst >>> index 1609a57d0..7cc99044a 100644 >>> --- a/doc/guides/prog_guide/lpm_lib.rst >>> +++ b/doc/guides/prog_guide/lpm_lib.rst >>> @@ -145,6 +145,38 @@ depending on whether we need to move to the next table or not. >>> Prefix expansion is one of the keys of this algorithm, >>> since it improves the speed dramatically by adding redundancy. >>> >>> +Deletion >>> +~~~~~~~~ >>> + >>> +When deleting a rule, a replacement rule is searched for. Replacement rule is an existing rule that has >>> +the longest prefix match with the rule to be deleted, but has smaller depth. >>> + >>> +If a replacement rule is found, target tbl24 and tbl8 entries are updated to have the same depth and next hop >>> +value with the replacement rule. >>> + >>> +If no replacement rule can be found, target tbl24 and tbl8 entries will be cleared. >>> + >>> +Prefix expansion is performed if the rule's depth is not exactly 24 bits or 32 bits. >>> + >>> +After deleting a rule, a group of tbl8s that belongs to the same tbl24 entry are freed in following cases: >>> + >>> +* All tbl8s in the group are empty . >>> + >>> +* All tbl8s in the group have the same values and with depth no greater than 24. >>> + >>> +Free of tbl8s have different behaviors: >>> + >>> +* If RCU is not used, tbl8s are cleared and reclaimed immediately. >>> + >>> +* If RCU is used, tbl8s are reclaimed when readers are in quiescent state. >>> + >>> +When the LPM is not using RCU, tbl8 group can be freed immediately even though the readers might be using >>> +the tbl8 group entries. This might result in incorrect lookup results. >>> + >>> +RCU QSBR process is integrated for safe tbl8 group reclaimation. Application has certain responsibilities >>> +while using this feature. Please refer to resource reclaimation framework of :ref:`RCU library ` >>> +for more details. >>> + >> >> Would the lpm6 library benefit from the same? >> Asking as I do not see much code shared between lpm and lpm6. >> >> [...] >> >>> diff --git a/lib/librte_lpm/rte_lpm.c b/lib/librte_lpm/rte_lpm.c >>> index 38ab512a4..41e9c49b8 100644 >>> --- a/lib/librte_lpm/rte_lpm.c >>> +++ b/lib/librte_lpm/rte_lpm.c >>> @@ -1,5 +1,6 @@ >>> /* SPDX-License-Identifier: BSD-3-Clause >>> * Copyright(c) 2010-2014 Intel Corporation >>> + * Copyright(c) 2020 Arm Limited >>> */ >>> >>> #include >>> @@ -245,13 +246,84 @@ rte_lpm_free(struct rte_lpm *lpm) >>> TAILQ_REMOVE(lpm_list, te, next); >>> >>> rte_mcfg_tailq_write_unlock(); >>> - >>> +#ifdef ALLOW_EXPERIMENTAL_API >>> + if (lpm->dq) >>> + rte_rcu_qsbr_dq_delete(lpm->dq); >>> +#endif >> >> All DPDK code under lib/ is compiled with the ALLOW_EXPERIMENTAL_API flag set. >> There is no need to protect against this flag in rte_lpm.c. >> >> [...] >> >>> diff --git a/lib/librte_lpm/rte_lpm.h b/lib/librte_lpm/rte_lpm.h >>> index b9d49ac87..7889f21b3 100644 >>> --- a/lib/librte_lpm/rte_lpm.h >>> +++ b/lib/librte_lpm/rte_lpm.h >> >>> @@ -130,6 +143,28 @@ struct rte_lpm { >>> __rte_cache_aligned; /**< LPM tbl24 table. */ >>> struct rte_lpm_tbl_entry *tbl8; /**< LPM tbl8 table. */ >>> struct rte_lpm_rule *rules_tbl; /**< LPM rules. */ >>> +#ifdef ALLOW_EXPERIMENTAL_API >>> + /* RCU config. */ >>> + struct rte_rcu_qsbr *v; /* RCU QSBR variable. */ >>> + enum rte_lpm_qsbr_mode rcu_mode;/* Blocking, defer queue. */ >>> + struct rte_rcu_qsbr_dq *dq; /* RCU QSBR defer queue. */ >>> +#endif >>> +}; >> >> This is more a comment/question for the lpm maintainers. >> >> Afaics, the rte_lpm structure is exported/public because of lookup >> which is inlined. >> But most of the structure can be hidden and stored in a private >> structure that would embed the exposed rte_lpm. >> The slowpath functions would only have to translate from publicly >> exposed to internal representation (via container_of). >> >> This patch could do this and be the first step to hide the unneeded >> exposure of other fields (later/in 20.11 ?). >> >> Thoughts? >> > Hiding as much of the structures as possible is always a good idea, so if > that is possible in this patchset I would support such a move. > > /Bruce > Agreed - I acked the change as it doesn't break ABI compatibility. Bruce and David's comments still hold for 20.11+. Ray K