From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 01FF0A0350; Mon, 29 Jun 2020 13:56:26 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B0B411BDFD; Mon, 29 Jun 2020 13:56:25 +0200 (CEST) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by dpdk.org (Postfix) with ESMTP id D5CD41B9BF for ; Mon, 29 Jun 2020 13:56:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593431783; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=isWJMH6Q1QNPYZbZCvVRDoOYkf965JsHmc50meUhis4=; b=fcLlD/P11tZQf9s1teQRMPawuAcvzr1B6nmoGJ7oiMSMP23Rdmim6YgPyI90SOPbzTDi+j UWd1y5low/QMdlrJUy8Ew5l9USajlADv8yCh8Q/oHiuOfCYUHDsfy00vr0nRiiepo8/XqD urGgHLBcmrO/4pOnPlKkT5ToAmBDx9s= Received: from mail-vs1-f70.google.com (mail-vs1-f70.google.com [209.85.217.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-89-JgbpYmdhMWqiNQKB0vnh6w-1; Mon, 29 Jun 2020 07:56:19 -0400 X-MC-Unique: JgbpYmdhMWqiNQKB0vnh6w-1 Received: by mail-vs1-f70.google.com with SMTP id x10so4943411vsj.22 for ; Mon, 29 Jun 2020 04:56:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=isWJMH6Q1QNPYZbZCvVRDoOYkf965JsHmc50meUhis4=; b=mGRn8h5ub7RuShWow7oY9A73JE/Oi5ww0sJDnpoJt95knNEcdozztwHpVywpoUrJZZ XDtsvF/gJ/DecTJeItzNWAMTKBWx6Ntq2sHg6CAjwjAS47QgIEkEqsMax12kEtu6tOD+ PmCTq+AL921ZjEiTeePVEaHA3vuFsgN3D/ceBuShleuCUGBABI5/o+Gn9pKNuXMJE4cM UDEeHM+phsL6I6Z7VBgSrPnU7fiC0IUxjLx1RPuy8ykTklSAP2wOdrcUnkOUImfp0mEm GeCB5DKSt8/kQhRugQiHkhUMQyyi4s7jgwFlWaU7fClH7KeTgqxztw7utgSGdmFVMv5R U8lw== X-Gm-Message-State: AOAM530tHogSO4l5u2gRzaq3QMy4VXKxBnIoqvsx0Um4mBVZHtC7j+7F MQRsBxJcVYKvvivBXDxyrE8Y7T1ufZAq/LMBJidUUF5thZ3gLTbVZ36e17KVKWXrAj1ApCAXzHe w8F0VdKHIUzK/7OcnaoE= X-Received: by 2002:a1f:6084:: with SMTP id u126mr9701760vkb.56.1593431778675; Mon, 29 Jun 2020 04:56:18 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwPeRNQpAcO6ddF4reB2/rkKBuKePhyX0HKofGl8LlU7gDgyEHIayfLhd4rbuEmr97zG8Em6/SaUkfkS9S0rs8= X-Received: by 2002:a1f:6084:: with SMTP id u126mr9701735vkb.56.1593431778320; Mon, 29 Jun 2020 04:56:18 -0700 (PDT) MIME-Version: 1.0 References: <20190906094534.36060-1-ruifeng.wang@arm.com> <20200629080301.97515-1-ruifeng.wang@arm.com> <20200629080301.97515-2-ruifeng.wang@arm.com> In-Reply-To: <20200629080301.97515-2-ruifeng.wang@arm.com> From: David Marchand Date: Mon, 29 Jun 2020 13:56:07 +0200 Message-ID: To: Ruifeng Wang , Vladimir Medvedkin , Bruce Richardson Cc: John McNamara , Marko Kovacevic , Ray Kinsella , Neil Horman , dev , "Ananyev, Konstantin" , Honnappa Nagarahalli , nd Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dmarchan@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH v5 1/3] lib/lpm: integrate RCU QSBR X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Mon, Jun 29, 2020 at 10:03 AM Ruifeng Wang wrote: > > Currently, the tbl8 group is freed even though the readers might be > using the tbl8 group entries. The freed tbl8 group can be reallocated > quickly. This results in incorrect lookup results. > > RCU QSBR process is integrated for safe tbl8 group reclaim. > Refer to RCU documentation to understand various aspects of > integrating RCU library into other libraries. > > Signed-off-by: Ruifeng Wang > Reviewed-by: Honnappa Nagarahalli > --- > doc/guides/prog_guide/lpm_lib.rst | 32 +++++++ > lib/librte_lpm/Makefile | 2 +- > lib/librte_lpm/meson.build | 1 + > lib/librte_lpm/rte_lpm.c | 129 ++++++++++++++++++++++++++--- > lib/librte_lpm/rte_lpm.h | 59 +++++++++++++ > lib/librte_lpm/rte_lpm_version.map | 6 ++ > 6 files changed, 216 insertions(+), 13 deletions(-) > > diff --git a/doc/guides/prog_guide/lpm_lib.rst b/doc/guides/prog_guide/lpm_lib.rst > index 1609a57d0..7cc99044a 100644 > --- a/doc/guides/prog_guide/lpm_lib.rst > +++ b/doc/guides/prog_guide/lpm_lib.rst > @@ -145,6 +145,38 @@ depending on whether we need to move to the next table or not. > Prefix expansion is one of the keys of this algorithm, > since it improves the speed dramatically by adding redundancy. > > +Deletion > +~~~~~~~~ > + > +When deleting a rule, a replacement rule is searched for. Replacement rule is an existing rule that has > +the longest prefix match with the rule to be deleted, but has smaller depth. > + > +If a replacement rule is found, target tbl24 and tbl8 entries are updated to have the same depth and next hop > +value with the replacement rule. > + > +If no replacement rule can be found, target tbl24 and tbl8 entries will be cleared. > + > +Prefix expansion is performed if the rule's depth is not exactly 24 bits or 32 bits. > + > +After deleting a rule, a group of tbl8s that belongs to the same tbl24 entry are freed in following cases: > + > +* All tbl8s in the group are empty . > + > +* All tbl8s in the group have the same values and with depth no greater than 24. > + > +Free of tbl8s have different behaviors: > + > +* If RCU is not used, tbl8s are cleared and reclaimed immediately. > + > +* If RCU is used, tbl8s are reclaimed when readers are in quiescent state. > + > +When the LPM is not using RCU, tbl8 group can be freed immediately even though the readers might be using > +the tbl8 group entries. This might result in incorrect lookup results. > + > +RCU QSBR process is integrated for safe tbl8 group reclaimation. Application has certain responsibilities > +while using this feature. Please refer to resource reclaimation framework of :ref:`RCU library ` > +for more details. > + Would the lpm6 library benefit from the same? Asking as I do not see much code shared between lpm and lpm6. [...] > diff --git a/lib/librte_lpm/rte_lpm.c b/lib/librte_lpm/rte_lpm.c > index 38ab512a4..41e9c49b8 100644 > --- a/lib/librte_lpm/rte_lpm.c > +++ b/lib/librte_lpm/rte_lpm.c > @@ -1,5 +1,6 @@ > /* SPDX-License-Identifier: BSD-3-Clause > * Copyright(c) 2010-2014 Intel Corporation > + * Copyright(c) 2020 Arm Limited > */ > > #include > @@ -245,13 +246,84 @@ rte_lpm_free(struct rte_lpm *lpm) > TAILQ_REMOVE(lpm_list, te, next); > > rte_mcfg_tailq_write_unlock(); > - > +#ifdef ALLOW_EXPERIMENTAL_API > + if (lpm->dq) > + rte_rcu_qsbr_dq_delete(lpm->dq); > +#endif All DPDK code under lib/ is compiled with the ALLOW_EXPERIMENTAL_API flag set. There is no need to protect against this flag in rte_lpm.c. [...] > diff --git a/lib/librte_lpm/rte_lpm.h b/lib/librte_lpm/rte_lpm.h > index b9d49ac87..7889f21b3 100644 > --- a/lib/librte_lpm/rte_lpm.h > +++ b/lib/librte_lpm/rte_lpm.h > @@ -130,6 +143,28 @@ struct rte_lpm { > __rte_cache_aligned; /**< LPM tbl24 table. */ > struct rte_lpm_tbl_entry *tbl8; /**< LPM tbl8 table. */ > struct rte_lpm_rule *rules_tbl; /**< LPM rules. */ > +#ifdef ALLOW_EXPERIMENTAL_API > + /* RCU config. */ > + struct rte_rcu_qsbr *v; /* RCU QSBR variable. */ > + enum rte_lpm_qsbr_mode rcu_mode;/* Blocking, defer queue. */ > + struct rte_rcu_qsbr_dq *dq; /* RCU QSBR defer queue. */ > +#endif > +}; This is more a comment/question for the lpm maintainers. Afaics, the rte_lpm structure is exported/public because of lookup which is inlined. But most of the structure can be hidden and stored in a private structure that would embed the exposed rte_lpm. The slowpath functions would only have to translate from publicly exposed to internal representation (via container_of). This patch could do this and be the first step to hide the unneeded exposure of other fields (later/in 20.11 ?). Thoughts? -- David Marchand