From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 39F9B42BF1; Mon, 5 Jun 2023 14:50:55 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0EFBE4021F; Mon, 5 Jun 2023 14:50:55 +0200 (CEST) Received: from mail-qt1-f169.google.com (mail-qt1-f169.google.com [209.85.160.169]) by mails.dpdk.org (Postfix) with ESMTP id A95924003C for ; Mon, 5 Jun 2023 14:50:53 +0200 (CEST) Received: by mail-qt1-f169.google.com with SMTP id d75a77b69052e-3f829e958bdso44038121cf.3 for ; Mon, 05 Jun 2023 05:50:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685969453; x=1688561453; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=ZfLj8Im5ShJlPY4x889ae/F1CjgjA3CmGx18jpcsU0Y=; b=hvaZ6/9mi9W0WKesQUb2/kqwxLGdePq+8YecQdcQHJ4x2YNqv7+0V6RuedNTQMTWgp b3OW0m6FeEF/DQEKAdFFbKyD6DMQiMSSM2aRhfklTcaoY6TEzoZyZ7Xyg+b53lq6whYH n/Q6SurTPCZaQEHY42vF4J3Fz1vLMJC7O0wMH7Ea8pPBYSenOQ9M9YfwNv71W38mmNrH PFu7r5/JsBnD8mXXgC9goF6p+19H0S72ZxsVLX2oBASBfQD0Cuw6rGgk3jNFVdunIFmY yybCHYNRNh4wgkvl3UX6/jBhqfP3LyJiFlJvb1NriMEN88MQdPfiFe3sG+f/uiDX9lLY gEKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685969453; x=1688561453; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZfLj8Im5ShJlPY4x889ae/F1CjgjA3CmGx18jpcsU0Y=; b=i5R7Kj5AcOewsNAxEBWPHlJskb2YJ/HbBAwch9DrwBBdSF74+woxGGMw8AG9Qm072G xbonfF4myXd+87fAuQDEWMv2poVHzN8WEVbnErrvSCh2u07EMVDb8zLItuHivkiSotwr VnVY3dECcU8DRa8b8i3zrtP455PfIP3sH7AYUgqP7pkDFQimMXJcuInnzU1h+2L7gnZd Sw2pn3PCecxW47yTWPScH16F8qhxsq99L66FvtWYXyIJ/jziTUZdo/CUdbXQDZQeuNA0 WsdmpKXCzUfNK9KFyDkI53DjkvS0gMLTl7ZEwhiiRY3YpI5p5KBP/sMmWX7kbwW2MUK6 hnvA== X-Gm-Message-State: AC+VfDz0iEmEW+/aM2+fb01zb5kS48WHV1XLJ3sIo5IKhs267eNXOqId qC3Uw9GaURq2K83ULfZ+ZMJeBmbseohzpNtCW1c= X-Google-Smtp-Source: ACHHUZ4B70NYsJWjvXP1zg+1pBcnt3g70qKOUFDClw5koI0P0apukqetbhAQwUaB4jyPkyXHOUVbOszNFiYCE0ukQas= X-Received: by 2002:ac8:5fca:0:b0:3f8:6924:a0b6 with SMTP id k10-20020ac85fca000000b003f86924a0b6mr8006969qta.50.1685969452926; Mon, 05 Jun 2023 05:50:52 -0700 (PDT) MIME-Version: 1.0 References: <20230425131516.3308612-1-vattunuru@marvell.com> <20230602162216.1868870-1-vattunuru@marvell.com> <20230602162216.1868870-3-vattunuru@marvell.com> In-Reply-To: <20230602162216.1868870-3-vattunuru@marvell.com> From: Nithin Dabilpuram Date: Mon, 5 Jun 2023 18:20:41 +0530 Message-ID: Subject: Re: [PATCH v3 2/3] node/kernel_rx: support receiving packets from kernel To: Vamsi Attunuru Cc: dev@dpdk.org, thomas@monjalon.net, jerinj@marvell.com, ndabilpuram@marvell.com, zhirun.yan@intel.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Acked-by: Nithin Dabilpuram On Fri, Jun 2, 2023 at 9:52=E2=80=AFPM Vamsi Attunuru wrote: > > Adds a node to receive packets from kernel over > a raw socket. > > Signed-off-by: Vamsi Attunuru > --- > doc/guides/prog_guide/graph_lib.rst | 8 + > lib/node/kernel_rx.c | 276 ++++++++++++++++++++++++++++ > lib/node/kernel_rx_priv.h | 48 +++++ > lib/node/meson.build | 1 + > 4 files changed, 333 insertions(+) > > diff --git a/doc/guides/prog_guide/graph_lib.rst b/doc/guides/prog_guide/= graph_lib.rst > index fa22b014f3..4b05bcee3c 100644 > --- a/doc/guides/prog_guide/graph_lib.rst > +++ b/doc/guides/prog_guide/graph_lib.rst > @@ -401,3 +401,11 @@ socket interface to transmit the packets, it uses th= e packet's destination > IP address in sockaddr_in address structure and ``sendto`` function to s= end > data on the raw socket. Aftering sending the burst of packets to kernel, > this node free up the packet buffers. > + > +kernel_rx > +~~~~~~~~~ > +This node is a source node which receives packets from kernel and forwar= ds to > +any of the intermediate nodes. It uses the raw socket interface to recei= ve > +packets from kernel. Uses ``poll`` function to poll on the socket fd for > +``POLLIN`` events to read the packets from raw socket to stream buffer a= nd does > +``rte_node_next_stream_move()`` when there are received packets. > diff --git a/lib/node/kernel_rx.c b/lib/node/kernel_rx.c > new file mode 100644 > index 0000000000..2dba7c8cc7 > --- /dev/null > +++ b/lib/node/kernel_rx.c > @@ -0,0 +1,276 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2023 Marvell International Ltd. > + */ > + > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include "ethdev_rx_priv.h" > +#include "kernel_rx_priv.h" > +#include "node_private.h" > + > +static inline struct rte_mbuf * > +alloc_rx_mbuf(kernel_rx_node_ctx_t *ctx) > +{ > + kernel_rx_info_t *rx =3D ctx->recv_info; > + > + if (rx->idx >=3D rx->cnt) { > + uint16_t cnt; > + > + rx->idx =3D 0; > + rx->cnt =3D 0; > + > + cnt =3D rte_pktmbuf_alloc_bulk(ctx->pktmbuf_pool, rx->rx_= bufs, KERN_RX_CACHE_COUNT); > + if (cnt <=3D 0) > + return NULL; > + > + rx->cnt =3D cnt; > + } > + > + return rx->rx_bufs[rx->idx++]; > +} > + > +static inline void > +mbuf_update(struct rte_mbuf **mbufs, uint16_t nb_pkts) > +{ > + struct rte_net_hdr_lens hdr_lens; > + struct rte_mbuf *m; > + int i; > + > + for (i =3D 0; i < nb_pkts; i++) { > + m =3D mbufs[i]; > + > + m->packet_type =3D rte_net_get_ptype(m, &hdr_lens, RTE_PT= YPE_ALL_MASK); > + > + m->ol_flags =3D 0; > + m->tx_offload =3D 0; > + > + m->l2_len =3D hdr_lens.l2_len; > + m->l3_len =3D hdr_lens.l3_len; > + m->l4_len =3D hdr_lens.l4_len; > + } > +} > + > +static uint16_t > +recv_pkt_parse(void **objs, uint16_t nb_pkts) > +{ > + uint16_t pkts_left =3D nb_pkts; > + struct rte_mbuf **pkts; > + int i; > + > + pkts =3D (struct rte_mbuf **)objs; > + > + if (pkts_left >=3D 4) { > + for (i =3D 0; i < 4; i++) > + rte_prefetch0(rte_pktmbuf_mtod(pkts[i], void *)); > + } > + > + while (pkts_left >=3D 12) { > + /* Prefetch next-next mbufs */ > + rte_prefetch0(pkts[8]); > + rte_prefetch0(pkts[9]); > + rte_prefetch0(pkts[10]); > + rte_prefetch0(pkts[11]); > + > + /* Prefetch next mbuf data */ > + rte_prefetch0(rte_pktmbuf_mtod(pkts[4], void *)); > + rte_prefetch0(rte_pktmbuf_mtod(pkts[5], void *)); > + rte_prefetch0(rte_pktmbuf_mtod(pkts[6], void *)); > + rte_prefetch0(rte_pktmbuf_mtod(pkts[7], void *)); > + > + /* Extract ptype of mbufs */ > + mbuf_update(pkts, 4); > + > + pkts +=3D 4; > + pkts_left -=3D 4; > + } > + > + if (pkts_left > 0) > + mbuf_update(pkts, pkts_left); > + > + return nb_pkts; > +} > + > +static uint16_t > +kernel_rx_node_do(struct rte_graph *graph, struct rte_node *node, kernel= _rx_node_ctx_t *ctx) > +{ > + kernel_rx_info_t *rx; > + uint16_t next_index; > + int fd; > + > + rx =3D ctx->recv_info; > + next_index =3D rx->node_next; > + > + fd =3D rx->sock; > + if (fd > 0) { > + struct rte_mbuf **mbufs; > + uint16_t len =3D 0, count =3D 0; > + int nb_cnt, i; > + > + nb_cnt =3D (node->size >=3D RTE_GRAPH_BURST_SIZE) ? RTE_G= RAPH_BURST_SIZE : node->size; > + > + mbufs =3D (struct rte_mbuf **)node->objs; > + for (i =3D 0; i < nb_cnt; i++) { > + struct rte_mbuf *m =3D alloc_rx_mbuf(ctx); > + > + if (!m) > + break; > + > + len =3D read(fd, rte_pktmbuf_mtod(m, char *), rte= _pktmbuf_tailroom(m)); > + if (len =3D=3D 0 || len =3D=3D 0xFFFF) { > + rte_pktmbuf_free(m); > + if (rx->idx <=3D 0) > + node_dbg("kernel_rx", "rx_mbuf ar= ray is empty\n"); > + rx->idx--; > + break; > + } > + *mbufs++ =3D m; > + > + m->port =3D node->id; > + rte_pktmbuf_data_len(m) =3D len; > + > + count++; > + } > + > + if (count) { > + recv_pkt_parse(node->objs, count); > + node->idx =3D count; > + > + /* Enqueue to next node */ > + rte_node_next_stream_move(graph, node, next_index= ); > + } > + > + return count; > + } > + > + return 0; > +} > + > +static uint16_t > +kernel_rx_node_process(struct rte_graph *graph, struct rte_node *node, v= oid **objs, > + uint16_t nb_objs) > +{ > + kernel_rx_node_ctx_t *ctx =3D (kernel_rx_node_ctx_t *)node->ctx; > + int fd; > + > + RTE_SET_USED(objs); > + RTE_SET_USED(nb_objs); > + > + if (!ctx) > + return 0; > + > + fd =3D ctx->recv_info->sock; > + if (fd > 0) { > + struct pollfd fds =3D {.fd =3D fd, .events =3D POLLIN}; > + > + if (poll(&fds, 1, 0) > 0) { > + if (fds.revents & POLLIN) > + return kernel_rx_node_do(graph, node, ctx= ); > + } > + } > + > + return 0; > +} > + > +static int > +kernel_rx_node_init(const struct rte_graph *graph, struct rte_node *node= ) > +{ > + struct kernel_rx_node_main *rx_node_main =3D kernel_rx_node_data_= get(); > + kernel_rx_node_ctx_t *ctx =3D (kernel_rx_node_ctx_t *)node->ctx; > + kernel_rx_node_elem_t *elem =3D rx_node_main->head; > + kernel_rx_info_t *recv_info; > + int sock; > + > + while (elem) { > + if (elem->nid =3D=3D node->id) { > + /* Update node specific context */ > + memcpy(ctx, &elem->ctx, sizeof(kernel_rx_node_ctx= _t)); > + break; > + } > + elem =3D elem->next; > + } > + > + RTE_VERIFY(elem !=3D NULL); > + > + if (ctx->pktmbuf_pool =3D=3D NULL) { > + node_err("kernel_rx", "Invalid mbuf pool on graph %s\n", = graph->name); > + return -EINVAL; > + } > + > + recv_info =3D rte_zmalloc_socket("kernel_rx_info", sizeof(kernel_= rx_info_t), > + RTE_CACHE_LINE_SIZE, graph->socket= ); > + if (!recv_info) { > + node_err("kernel_rx", "Kernel recv_info is NULL\n"); > + return -ENOMEM; > + } > + > + sock =3D socket(AF_INET, SOCK_RAW, IPPROTO_RAW); > + if (sock < 0) { > + node_err("kernel_rx", "Unable to open RAW socket\n"); > + return sock; > + } > + > + recv_info->sock =3D sock; > + ctx->recv_info =3D recv_info; > + > + return 0; > +} > + > +static void > +kernel_rx_node_fini(const struct rte_graph *graph __rte_unused, struct r= te_node *node) > +{ > + kernel_rx_node_ctx_t *ctx =3D (kernel_rx_node_ctx_t *)node->ctx; > + > + if (ctx->recv_info) { > + close(ctx->recv_info->sock); > + ctx->recv_info->sock =3D -1; > + rte_free(ctx->recv_info); > + } > + > + ctx->recv_info =3D NULL; > +} > + > +struct kernel_rx_node_main * > +kernel_rx_node_data_get(void) > +{ > + static struct kernel_rx_node_main kernel_rx_main; > + > + return &kernel_rx_main; > +} > + > +static struct rte_node_register kernel_rx_node_base =3D { > + .process =3D kernel_rx_node_process, > + .flags =3D RTE_NODE_SOURCE_F, > + .name =3D "kernel_rx", > + > + .init =3D kernel_rx_node_init, > + .fini =3D kernel_rx_node_fini, > + > + .nb_edges =3D KERNEL_RX_NEXT_MAX, > + .next_nodes =3D { > + [KERNEL_RX_NEXT_PKT_CLS] =3D "pkt_cls", > + [KERNEL_RX_NEXT_IP4_LOOKUP] =3D "ip4_lookup", > + }, > +}; > + > +struct rte_node_register * > +kernel_rx_node_get(void) > +{ > + return &kernel_rx_node_base; > +} > + > +RTE_NODE_REGISTER(kernel_rx_node_base); > diff --git a/lib/node/kernel_rx_priv.h b/lib/node/kernel_rx_priv.h > new file mode 100644 > index 0000000000..f1aa2344d7 > --- /dev/null > +++ b/lib/node/kernel_rx_priv.h > @@ -0,0 +1,48 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2023 Marvell International Ltd. > + */ > + > +#ifndef __KERNEL_RX_PRIV_H__ > +#define __KERNEL_RX_PRIV_H__ > + > +#define KERN_RX_CACHE_COUNT 64 > + > +typedef struct kernel_rx_info { > + struct rte_mbuf *rx_bufs[KERN_RX_CACHE_COUNT]; > + uint16_t node_next; > + uint16_t idx; > + uint16_t cnt; > + int sock; > +} kernel_rx_info_t; > + > +/* kernel_rx node context structure */ > +typedef struct kernel_rx_node_ctx { > + struct rte_mempool *pktmbuf_pool; > + kernel_rx_info_t *recv_info; > +} kernel_rx_node_ctx_t; > + > +/* kernel_rx node list element structure */ > +typedef struct kernel_rx_node_elem { > + struct kernel_rx_node_elem *next; /* Pointer to the next node ele= ment. */ > + struct kernel_rx_node_ctx ctx; /* kernel_rx node context. */ > + rte_node_t nid; /* Node identifier of the kerne= l_rx node. */ > +} kernel_rx_node_elem_t; > + > +enum kernel_rx_next_nodes { > + KERNEL_RX_NEXT_PKT_CLS, > + KERNEL_RX_NEXT_IP4_LOOKUP, > + KERNEL_RX_NEXT_MAX, > +}; > + > +/* kernel_rx node main structure */ > +struct kernel_rx_node_main { > + kernel_rx_node_elem_t *head; /* Pointer to the head node element.= */ > +}; > + > +/* Get the pointer of kernel_rx node data */ > +struct kernel_rx_node_main *kernel_rx_node_data_get(void); > + > +/* Get the pointer of kernel_rx node register structure */ > +struct rte_node_register *kernel_rx_node_get(void); > + > +#endif /* __KERNEL_RX_PRIV_H__ */ > diff --git a/lib/node/meson.build b/lib/node/meson.build > index 0520be23ff..1f04fd1d7e 100644 > --- a/lib/node/meson.build > +++ b/lib/node/meson.build > @@ -13,6 +13,7 @@ sources =3D files( > 'ethdev_tx.c', > 'ip4_lookup.c', > 'ip4_rewrite.c', > + 'kernel_rx.c', > 'kernel_tx.c', > 'log.c', > 'null.c', > -- > 2.25.1 >