From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 64CD045B38; Mon, 14 Oct 2024 16:34:53 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8024F40696; Mon, 14 Oct 2024 16:34:32 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 4578940696 for ; Mon, 14 Oct 2024 16:34:30 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49E9k6iW015672; Mon, 14 Oct 2024 07:34:24 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pfpt0220; bh=R QbcQ9e4eNvbho7yOGOE347wUXQmweVXn3h8td8qVj4=; b=E5wEo38cTT+SVtUiY XdSxS46MbLWX4bKzeDtEbh8O/h7yeMYVSGV1hTeGjf1MW73CvVazmwiQZyVp7Zh5 QEvodowr9U3Cc+LodlMDBEsiKtT3impgY6Zfe58Fw6C3BxrH5W4KyZ5otsreLpDg adm003rngrc6D1vFMmOm7VKvyd8CQe2fIjaodn54Nx9mhd6kS7SH5uI+4zxvul2F RZuGKX5F2CIC/bvK0Eqd8cXdKl4NLnjnrOM4dTSrmGAC2xk2w9VGXwDE3tqejLes 9Qrgt+/cd1oOy5PR/J+p3hRhOL2cCtsdNnEmXgF4YKync1eSa32ICMHd2ysftgZs stFOA== Received: from dc5-exch05.marvell.com ([199.233.59.128]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 4290y70gct-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 14 Oct 2024 07:34:23 -0700 (PDT) Received: from DC5-EXCH05.marvell.com (10.69.176.209) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 14 Oct 2024 07:34:21 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH05.marvell.com (10.69.176.209) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Mon, 14 Oct 2024 07:34:21 -0700 Received: from cavium-PowerEdge-R640.. (unknown [10.28.36.207]) by maili.marvell.com (Postfix) with ESMTP id 87F583F7055; Mon, 14 Oct 2024 07:34:18 -0700 (PDT) From: Nitin Saxena To: Jerin Jacob , Kiran Kumar K , Nithin Dabilpuram , Zhirun Yan , Robin Jarry , Christophe Fontaine CC: , Nitin Saxena Subject: [PATCH v5 5/5] docs: add programming guide for feature arc Date: Mon, 14 Oct 2024 20:03:58 +0530 Message-ID: <20241014143401.3135897-6-nsaxena@marvell.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014143401.3135897-1-nsaxena@marvell.com> References: <20241010133111.2764712-1-nsaxena@marvell.com> <20241014143401.3135897-1-nsaxena@marvell.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Proofpoint-GUID: eE88NLTWkeshvvudNNUJ2suMPv_X5LhQ X-Proofpoint-ORIG-GUID: eE88NLTWkeshvvudNNUJ2suMPv_X5LhQ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-06_09,2024-09-06_01,2024-09-02_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Updated graph library guide with feature arc Signed-off-by: Nitin Saxena --- doc/guides/prog_guide/graph_lib.rst | 288 +++++++++++ doc/guides/prog_guide/img/feature_arc-1.svg | 277 +++++++++++ doc/guides/prog_guide/img/feature_arc-2.svg | 511 ++++++++++++++++++++ doc/guides/prog_guide/img/feature_arc-3.svg | 318 ++++++++++++ 4 files changed, 1394 insertions(+) create mode 100644 doc/guides/prog_guide/img/feature_arc-1.svg create mode 100644 doc/guides/prog_guide/img/feature_arc-2.svg create mode 100644 doc/guides/prog_guide/img/feature_arc-3.svg diff --git a/doc/guides/prog_guide/graph_lib.rst b/doc/guides/prog_guide/gr= aph_lib.rst index ad09bdfe26..d98b48b0e5 100644 --- a/doc/guides/prog_guide/graph_lib.rst +++ b/doc/guides/prog_guide/graph_lib.rst @@ -547,3 +547,291 @@ on success packet is enqueued to ``udp4_input`` node. =20 Hash lookup is performed in ``udp4_input`` node with registered destinatio= n port and destination port in UDP packet , on success packet is handed to ``udp_= user_node``. + +Feature Arc +----------- +`Feature arc` represents an ordered list of `protocols/features` at a given +networking layer. It is a high level abstraction to connect various `featu= re` +nodes in `rte_graph` instance and allows seamless packets steering based o= n the +sequence of enabled features at runtime on each interface. + +`Features` (or feature nodes) are nodes which handle partial or complete +protocol processing in a given direction. For instance, `ipv4-rewrite` and +`IPv4 IPsec encryption` are outbound features while `ipv4-lookup` and `IPv4 +IPsec decryption` are inbound features. Further, `ipv4-rewrite` and `IPv4 +IPsec encryption` can collectively represent a `feature arc` towards egress +direction with ordering constraints that `IPv4 IPsec encryption` must be +performed before `ipv4-rewrite`. Similarly, `IPv4 IPsec decryption` and +`ipv4-lookup` can represent a `feature arc` in an ingress direction. Both = of +these `feature arc` can co-exist at an IPv4 layer in egress and ingress +direction respectively. + +A `feature` can be represented by a single node or collection of multiple = nodes +performing feature processing collectively. + +.. figure:: img/feature_arc-1.* + :alt: feature-arc-1 + :width: 350px + :align: center + + Feature Arc overview + +Each `feature arc` is associated with a `Start` node from which all featur= es in +a feature arc are connected. A `start` node itself is not a `feature` nod= e but +it is where `first enabled feature` is checked in fast path. In above figu= re, +`Node-A` represents a `start node`. There may be a `Sink` node as well wh= ich +is child node for every feature in an arc. 'Sink` node is responsible of +consuming those packets which are not consumed by intermediate enabled fea= tures +between `start` and `sink` node. `Sink` node, if present, is the last enab= led +feature in a feature arc. A `feature` node statically connected to `start`= node +must also be added via feature arc API, `rte_graph_feature_add()``. Here `= Node-B` +acts as a `sink` node which is statically linked to `Node A`. `Feature` no= des +are connected via `rte_graph_feature_add()` which takes care of connecting +all `feature` nodes with each other and start node. + +.. code-block:: bash + :linenos: + :emphasize-lines: 8 + :caption: Node-B statically linked to Node-A + + static struct rte_node_register node_A_node =3D { + .process =3D node_A_process_func, + ... + ... + .name =3D "Node-A", + .next_nodes =3D + { + [0] =3D "Node-B", + }, + .nb_edges =3D 1, + }; + +When multiple features are enabled on an interface, it may be required to = steer +packets across `features` in a given order. For instance, if `Feature 1` a= nd +`Feature 2` both are enabled on an interface ``X``, it may be required to = send +packets to `Feature-1` before `Feature-2`. Such ordering constraints can be +easily expressed with `feature arc`. In this case, `Feature 1` is called as +``First Feature`` and `Feature 2` is called as ``Next Feature`` to `Featur= e 1`. + +.. figure:: img/feature_arc-2.* + :alt: feature-arc-2 + :width: 600px + :align: center + + First and Next features and their ordering + +In similar manner, even application specific ``custom features`` can be ho= oked +to standard nodes. It is to be noted that this `custom feature` hooking to +`feature arc` aware node does not require any code changes. + +It may be obvious by now that `features` enabled on one interface does not +affect packets on other interfaces. In above example, if no feature is +enabled on an interface ``X``, packets destined to interface ``X`` would be +directly sent to `Node-B` from `Node-A`. + +.. figure:: img/feature_arc-3.* + :alt: feature-arc-3 + :width: 550px + :align: center + + Feature-2 consumed/non-consumed packet path + +When a `Feature-X` node receives packets via feature arc, it may decide wh= ether +to ``consume packet`` or send to `next enabled feature`. A node can consume +packet by freeing it, sending it on wire or enqueuing it to hardware queue= . If a +packet is not consumed by a `Feature-X` node, it may send to `next enabled +feature` on an interface. In above figure, `Feature-2` nodes are represent= ed to +consume packets. Classic example for a node performing consume and non-con= sume +operation on packets would be IPsec policy node where all packets with +``protect`` actions are consumed while remaining packets with ``bypass`` +actions are sent to next enabled feature. + +In fast path feature node may require to lookup local data structures for = each +interface. For example, retrieving policy database per interface for IPsec +processing. ``rte_graph_feature_enable`` API allows to set application +specific cookie per feature per interface. `Feature data` object maintains= this +cookie in fast path for each interface. + +`Feature arc design` allows to enable subsequent features in a control pla= ne +without stopping workers which are accessing feature arc's fast path APIs = in +``rte_graph_walk()`` context. However for disabling features require RCU l= ike +scheme for synchronization. + +Programming model +~~~~~~~~~~~~~~~~~ +Feature Arc Objects +^^^^^^^^^^^^^^^^^^^ +Control plane and fast path APIs deals with following objects: + +Feature arc +*********** +``rte_graph_feature_arc_t`` is a handle to feature arc which is created via +``rte_graph_feature_arc_create()``. It is a `uint64_t` size object which c= an be +saved in feature node's context. This object can be translated to fast path +feature arc object ``struct rte_graph_feature_arc`` which is an input +argument to all fast path APIs. Control plane APIs majorly takes +`rte_graph_feature_arc_t` object as an input. + +Feature List +************ +Each feature arc holds two feature lists: `active` and `passive`. While wo= rker +cores uses `active` list, control plane APIs uses `passive` list for +enabling/disabling a feature on any interface with in a arc. After success= ful +feature enable/disable, ``rte_graph_feature_enable()``/ +``rte_graph_feature_disable()`` atomically switches passive list to active= list +and vice-versa. Most of the fast path APIs takes active list as an argument +(``rte_graph_feature_rt_list_t``), which feature node can obtain in start = of +it's `process_func()` via ``rte_graph_feature_arc_has_any_feature()`` (in = `start` +node) or ``rte_graph_feature_arc_has_feature()`` (in next feature nodes). + +Each feature list holds RTE_GRAPH_MAX_FEATURES number of features and +associated feature data for every interface index + +Feature +******** +Feature is a data structure which holds `feature data` object for every +interface. It is represented via ``rte_graph_feature_t`` which is a `uint= 8_t` +size object. Fast path internal structure ``struct rte_graph_feature`` can= be +obtained from ``rte_graph_feature_t`` via ``rte_graph_feature_get()`` API. + +In `start` node `rte_graph_feature_arc_first_feature_get()` can be used to= get +first enabled `rte_graph_feature_t` object for an interface. `rte_edge` fr= om +`start` node to first enabled feature is provided by +``rte_graph_feature_arc_feature_set()`` API. + +In `feature nodes`, next enabled feature is obtained by providing current = feature +as an input to ``rte_graph_feature_arc_next_feature_get()`` API. + +Feature data +************ +Feature data object is maintained per feature per interface which holds +following information in fast path + +- ``rte_edge_t`` to send packet to next enabled feature +- ``Next enabled feature`` on current interface +- ``User_data`` per feature per interface set by application via `rte_grap= h_feature_enable()` + +Enabling Feature Arc processing +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +By default, feature arc processing is disabled in `rte_graph_create()`. To +enable feature arc processing in fast path, `rte_graph_create()` API shoul= d be +invoked with `feature_arc_enable` flag set as `true` + +.. code-block:: bash + :linenos: + :emphasize-lines: 3 + :caption: Enabling feature are processing in rte_graph_create() + + struct rte_graph_param graph_conf; + + graph_conf.feature_arc_enable =3D true; + struct rte_graph *graph =3D rte_graph_create("graph_name", &graph_conf= ); + +Further as an optimization technique, `rte_graph_walk()` would call newly = added +``feat_arc_proc()`` node callback function (if non-NULL) instead of +``process`` + +.. code-block:: bash + :linenos: + :emphasize-lines: 3 + :caption: Feature arc specific node callback function + + static struct rte_node_register ip4_rewrite_node =3D { + .process =3D ip4_rewrite_node_process, + .feat_arc_proc =3D ip4_rewrite_feature_node_process, + .name =3D "ip4_rewrite", + ... + ... + }; + +If `feat_arc_proc` is not provided in node registration, `process_func` wo= uld +be called by `rte_graph_walk()` + +Sample Usage +^^^^^^^^^^^^ +.. code-block:: bash + :linenos: + :caption: Feature arc sample usage + + #define MAX_FEATURES 10 + #define MAX_INDEXES 5 + + static uint16_t + feature2_feature_node_process (struct rte_graph *graph, struct + rte_node *node, void **objs, uint16_t nb_= objs) + { + /* features may be enabled */ + } + static uint16_t + feature2_node_process (struct rte_graph *graph, struct + rte_node *node, void **objs, uint16_t nb_= objs) + { + /* Feature arc is disabled in rte_graph_create() */ + } + + static uint16_t + feature2_node_process (struct rte_graph *graph, struct + rte_node *node, void **objs, uint16_t nb_= objs) + { + /* Feature arc may be enabled or disabled as this process_func() wo= uld + * be called for the case when feature arc is enabled in rte_graph_= create() + * and also the case when it is disabled + */ + } + + static struct rte_node_register feature2_node =3D { + .process =3D feature2_node_process, + .feat_arc_proc =3D feature2_feature_node_process, + .name =3D "feature2", + .init =3D feature2_init_func, + ... + ... + }; + + static struct rte_node_register feature1_node =3D { + .process =3D feature1_node_process, + .feat_arc_proc =3D NULL, + .name =3D "feature1", + ... + ... + }; + + int worker_cb(void *_em) + { + rte_graph_feature_arc_t arc; + uint32_t user_data; + + rte_graph_feature_arc_lookup_by_name("sample_arc", &arc); + + /* From control thread context (like CLII): + * Enable feature 2 on interface index 4 + */ + if (rte_lcore_id() =3D=3D rte_get_main_lcore) { + user_data =3D 0x1234; + rte_graph_feature_enable(arc, 4 /* interface index */, "fe= ature2", user_data); + } else { + while(1) + rte_graph_walk); + } + } + + int main(void) + { + struct rte_graph_param graph_conf; + rte_graph_feature_arc_t arc; + + if (rte_graph_feature_arc_create("sample_arc", MAX_FEATURES, MAX_IN= DEXES, &arc)) + return -1; + + rte_graph_feature_add(arc, "feature1", NULL, NULL); + rte_graph_feature_add(arc, "feature2", "feature1" /* add feature2 a= fter feature 1*/, NULL); + + /* create graph*/ + ... + ... + graph_conf.feature_arc_enable =3D true; + + struct rte_graph *graph =3D rte_graph_create("sample_graph", &graph= _conf); + + rte_eal_mp_remote_launch(worker_cb, arg, CALL_MAIN); + } diff --git a/doc/guides/prog_guide/img/feature_arc-1.svg b/doc/guides/prog_= guide/img/feature_arc-1.svg new file mode 100644 index 0000000000..dada169e11 --- /dev/null +++ b/doc/guides/prog_guide/img/feature_arc-1.svg @@ -0,0 +1,277 @@ + + + + +image/svg+xmlNode +- +A +Node +- +B +Feature +- +1 +Static Packet Path +edge =3D=3D 0 +edge =3D=3D 1 +Source/Start +Node +End/Sink +Node +Feature +Node +Packet path gets activated in +Node +- +A +when Feature +- +1 is +enabled on +an interface + diff --git a/doc/guides/prog_guide/img/feature_arc-2.svg b/doc/guides/prog_= guide/img/feature_arc-2.svg new file mode 100644 index 0000000000..eb5fb37e04 --- /dev/null +++ b/doc/guides/prog_guide/img/feature_arc-2.svg @@ -0,0 +1,511 @@ + + + + +image/svg+xmlNode +- +A +Node +- +B +Feature +- +1 +Static Packet +Path +edge =3D=3D 0 +edge =3D=3D 1 +Custom +=E2=80=93 +Feature +edge =3D=3D 3 +Feature +- +2 +edge =3D=3D 2 +First Feature +For Interface +=E2=80=93 +0,1 +edge =3D=3D 1 +Next Feature to Feature +- +1 +for Interface +- +0 +edge =3D=3D 0 +Next Feature to Feature +- +1 +for Interface +- +1 +edge =3D=3D 0 +Interface +- +0 +Enabled Features +Interface +- +1 +Enabled Features +edge =3D=3D 2 +edge =3D=3D 0 +edge =3D=3D 0 + diff --git a/doc/guides/prog_guide/img/feature_arc-3.svg b/doc/guides/prog_= guide/img/feature_arc-3.svg new file mode 100644 index 0000000000..382f259e42 --- /dev/null +++ b/doc/guides/prog_guide/img/feature_arc-3.svg @@ -0,0 +1,318 @@ + + + + +image/svg+xmlNode +- +A +Node +- +B +Feature +- +1 +Feature +- +3 +Feature +- +2 +Not consumed +Feature +- +2b +Feature +- +2c +Consumed +rte_edge =3D=3D 3 +edge =3D=3D 0 +rte_edge =3D=3D 2 +rte_edge =3D=3D 1 +rte_edge =3D=3D 0 +Static Path + --=20 2.43.0