From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 70F1B45B5D; Thu, 17 Oct 2024 13:13:56 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E4B8040279; Thu, 17 Oct 2024 13:13:55 +0200 (CEST) Received: from mail-lf1-f46.google.com (mail-lf1-f46.google.com [209.85.167.46]) by mails.dpdk.org (Postfix) with ESMTP id D301C40279 for ; Thu, 17 Oct 2024 12:56:46 +0200 (CEST) Received: by mail-lf1-f46.google.com with SMTP id 2adb3069b0e04-539f72c913aso971190e87.1 for ; Thu, 17 Oct 2024 03:56:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729162606; x=1729767406; darn=dpdk.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=UfEccU7cxfVsS9bCuPm7nNyhyKLRQHJknCz07jtU0JA=; b=FQer16NDq4CrUjoS9Tvu9tSiRFaO53IS6uugjMWKNq0g/E+wb/MntvwEb155G3TlKp pKl0bYGY+/fUGbRKsxy2grj2MAEB1N1QtMcvLR9Y0L7+QcURZxNYHweyN+DPwlvLbanZ lezHemPxbIcqO0z4W/oOOKZImHyX/ZpHDD+vWw8mR3YfgQP+9Z6J+0TF9CA7fj70Zltn iEF0YGgxzxCVUT+LSFBW4dlDTdDZ/N1D8zci7qp+J8dgLjjo4bU9AWzDfbmD787oacBe EPBT8x5qzXVNUHGjz2ufglXdhaCjsZVACUEAXk2qzqKZUdxFVSV1R+tniacefNiAX/nL ltww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729162606; x=1729767406; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UfEccU7cxfVsS9bCuPm7nNyhyKLRQHJknCz07jtU0JA=; b=Sfy2LpKRHymeEdObcrp3tNQcrWjU6Dvq/cwOVBO1Ffiud9pReg9VDGxc5zaiwwGZcJ RWpU16xHGNpE6gww6jhG0Jth9dz2hLdgJZadTDygE9ebVveqXC3VMrAbAxgp3/ByoKUK /uGMr5T6MiCegJjTewE6CnNNT8fuMf54P9+75A1sp9XJanfw73XqPO1bbBvXoiyrxTi5 NS5LO1v9t8rZD3sbrzjMz6U8OVYox9Tw2yIXwvWD0r9Emxami5ZglANWfDxS1HwBEx1i Ds6MWCe/9pjLlhGA/O8cMy34/TJta0Iwi8RUMjy+uJwMEWUvX5+IHF1NDscwlQ09SbtJ tEYw== X-Forwarded-Encrypted: i=1; AJvYcCVQEKRTREqVpbBRKQFscxYp8tyuvT+zWJP1yEHi8cJC5P0AdQAVrKqfnRVpJfJFQjuwlrw=@dpdk.org X-Gm-Message-State: AOJu0YyLahZQ167SGdVtukNJYA7/RXNy406keZJDkUeUzilmVXg4e2l6 aIOxMN7L9ioxi2TxAS04mhwtvJ3dlWPiDaRFAU+A68OVmYzx2DloF4T/AGILbaPaJ+kCKIV5qKT Vb6YoLEre8dTefLIgijfKKdLPIajecGEpU/Y7sA== X-Google-Smtp-Source: AGHT+IH1yRopr4MyWc35OK/maz5/pf61G2MdwvIdCHXwX81MPJ8Sy9iaHffuUX/d2ni0Z2dM4pdAayQ0lIKqbRKxccA= X-Received: by 2002:a05:6512:118e:b0:52e:9b9e:a6cb with SMTP id 2adb3069b0e04-53a03f196f9mr4852923e87.15.1729162605771; Thu, 17 Oct 2024 03:56:45 -0700 (PDT) MIME-Version: 1.0 References: <20240907073115.1531070-1-nsaxena@marvell.com> <509C9EDF-701F-4F7F-9E08-462FD378FF1B@redhat.com> In-Reply-To: <509C9EDF-701F-4F7F-9E08-462FD378FF1B@redhat.com> From: Nitin Saxena Date: Thu, 17 Oct 2024 16:26:33 +0530 Message-ID: Subject: Re: [EXTERNAL] [RFC PATCH 0/3] add feature arc in rte_graph To: Christophe Fontaine Cc: Robin Jarry , David Marchand , Nitin Saxena , Jerin Jacob , Kiran Kumar Kokkilagadda , Nithin Kumar Dabilpuram , Zhirun Yan , "dev@dpdk.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Thu, 17 Oct 2024 13:13:55 +0200 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Christophe, Please see inline comments Thanks, Nitin On Thu, Oct 17, 2024 at 2:02=E2=80=AFPM Christophe Fontaine wrote: > > Hi all, > > What about the following steps: > - update the nodes so they work on the current layer (example: for all L3= nodes, the current mbuf data offset *must* be pointing to the IP header) Agreed. It would be better if nodes uses rte_pktmbuf_[append()/shrink() etc..] APIs to manipulate layer data offset > - define a public data structure that would be shared across nodes throug= h priv data, and not dynfields ? Eventually public data structures should be defined to serve *a purpose*. Do you refer to creating a generic public structure? If yes, IMO, it may not be tuned for performance IMO, we need to create public structures for each specific purpose. Feature arc is also a public data structure which optimally saves following variables in 8 byte compact structure (rte_graph_feature_daa_t) for every interface - rte_edge_t (uint16_t) - next enabled feature (uint8_t) per index (from current node) - Per interface feature specific user_data (uint32_t) Due to its compact nature, 8 such objects per interface can be saved in one 64B cache line. So IMO, it is better to create public structures for a given purpose and optimally define them fields and APIs. Also feature arc specific 3B data is saved in mbuf dynfield. Hard to say if priv data would provide a better solution. > This structure would be the "internal api" (so, that has to be tracked ac= ross dpdk releases) between nodes. > We=E2=80=99d need common data shared for all the nodes as well as specifi= c data between 2 nodes. > As we get to this point, this (hopefully) will help with the node reusabi= lity. Feature arc also maintains data between 2 nodes per interface and also for all nodes which are added as features. > > - Update the feature arcs to leverage this well known structure, and refi= ne the api > - Define which part of the stack needs to be defined as a feature arc, wi= th the benefit of the generic API to enable/disable that feature, and which= part needs to be dynamically pluggable. > For instance, for a router, it may not make sense to define IPv4 support = as a feature arc. > So, we=E2=80=99d statically connect eth_input to ip_input. Agreed > Yet, lldp support is a good candidate for a feature arc: we need to confi= gure it per interface, and this is independent of the main graph. > There would be more protocols which need to be enabled per interface > WDYT? > Christophe > > > On 17 Oct 2024, at 09:50, Robin Jarry wrote: > > > > Hi Nitin, all, > > > > Nitin Saxena, Oct 17, 2024 at 09:03: > >> Hi Robin/David and all, > >> > >> We realized the feature arc patch series is difficult to understand as= a new concept. Our objectives are following with feature arc changes > >> > >> 1. Allow reusability of standard DPDK nodes (defined in lib/nodes/*) = with out-of-tree applications (like grout). Currently out-of-tree grap= h applications are duplicating standard nodes but not reusing the standa= rd ones which are available. In the long term, we would like to mature s= tandard DPDK nodes with flexibility of hooking them to out-of-tree appli= cation nodes. > > > > It would be ideal if the in-built nodes could be reused. When we starte= d working on grout, I tried multiple approaches where I could reuse these n= odes, but all failed. The nodes public API seems tailored for app/graph but= does not fit well with other control plane implementations. > > > > One of the main issues I had is that the ethdev_rx and ethdev_tx nodes = are cloned per rxq / txq associated with a graph worker. The rte_node API r= equires that every clone has a unique name. This in turn makes hot plugging= of DPDK ports very complex, if not impossible. > > > > For example, with the in-built nodes, it is not possible to change the = number of ports or their number of RX queues without destroying the whole g= raph and creating a new one from scratch. > > > > Also, the current implementation of "ip{4,6}-rewrite" handles writing e= thernet header data. This would prevent it from using this node for an IP-i= n-IP tunnel interface as we did in grout. > > > > Do you think we could change the in-built nodes to enforce OSI layer se= paration of concerns? It would make them much more flexible. It may cause a= slight drop of performance because you'd be splitting processing in two di= fferent nodes. But I think flexibility is more important. Otherwise, the in= -built nodes can only be used for very specific use-cases. > > > > Finally, I would like to improve the rte_node API to allow defining and= enforcing per-packet metadata that every node expects as input. The curren= t in-built nodes rely on mbuf dynamic fields for this but this means you on= ly have 9x32 bits available. And using all of these may break some drivers = (ixgbe) that rely on dynfields to work. Have you considered using mbuf priv= ate data for this? > > > >> > >> 2. Flexibility to enable/disable sub-graphs per interface based on the= runtime configuration updates. Protocol sub-graphs can be selectivel= y enabled for few (or all interfaces) at runtime > >> > >> 3. More than one sub-graphs/features can be enabled on an interface. = So a packet has to follow a sequential ordering node path on worker co= res. Packets may need to move from one sub-graph to another sub-graph pe= r interface > >> > >> 4. Last but not least, an optimized implementation which does not (or = minimally) stop worker cores for any control plane runtime updates. A= ny performance regression should also be avoided > >> > >> I am planning to create a draft presentation on feature arc which I ca= n share, when ready, to discuss. If needed, I can also plan to present that= in one of the DPDK community meetings. Their we can also discuss if there = are any alternatives of achieving above objectives > > > > Looking forward to this. > > > > Thanks! > > >