From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1B5B043407 for ; Wed, 29 Nov 2023 23:22:12 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D0677402E6; Wed, 29 Nov 2023 23:22:11 +0100 (CET) Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) by mails.dpdk.org (Postfix) with ESMTP id A1B49402E1 for ; Wed, 29 Nov 2023 23:22:09 +0100 (CET) Received: by mail-ej1-f53.google.com with SMTP id a640c23a62f3a-a1882023bbfso984966b.3 for ; Wed, 29 Nov 2023 14:22:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ochsnet-com.20230601.gappssmtp.com; s=20230601; t=1701296529; x=1701901329; darn=dpdk.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=8zHvXeGVGvTQ1wf69y7D7/6UvGtDKJfKszHWIFkSmsQ=; b=phvt1OGWVvhrOEoWNnLOTmO/D2J+8mLx6lfZjyv3+zBar2lZlApIlukXGKGpZmrXX+ Hx2sjCY+9Vo/8kUg2AvzIppFSfwUvn++TeKwOa+ZCEx3imzHPb5tmyomEH9iPPQQZlEM nILKp1N3X1ZbS2d5u+bpMj7qXkOxZCb7KHyPLHBZ8V4LzcZh/oMhAzyQeQc9PT6yoKTE N9Z16L5TZkOTJff05pdbjU01rOO17bICmq3q1F3dvLscRce/wOtpBOP5II+jKWvvZYzX Z2Ym8PKrYp5zs1V0+Yr8Q1bazKsbeu4zJyOqYnAGYkEKQBz9D0aj0l6Ktzf33gznmKQe 29kw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701296529; x=1701901329; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=8zHvXeGVGvTQ1wf69y7D7/6UvGtDKJfKszHWIFkSmsQ=; b=EqdJdwh7Wu83IqKuN59saS+22kkHhPV24WDhgZhrxhgRPJAmrlREXEMu5MVr8L9Gxj rLIwodIIpLr3bfYtz5JEiBKhmjXnl/85q7iGV0vguBcnVLPTPIydjmUf6eHLO4YoNnRI Xja9mPSot0AiWU9/pmw1O/IL9RVZRGrGYcUQERNQSzDpZLV5sissGQ7Q5hv9TURBYvPF JTmNwp/iJBSjK7ZVkhcHdSACflLKF1IGl1xAlQC0RRdSGe5dr2ZK0Yyzi+vH84EDafNs 6E2BhewZzhzaTNr70YIEye69JLgC73x6EnwIgIBMRNvW+h5iALX4cEXXDSgXB1h5jwlC wGrQ== X-Gm-Message-State: AOJu0Yzlo26rCG8YmhgTixfkUtnZtIigQ9AjX7wTjQnC+P6kbk3DhRWx SDBxRptlE91Y9BZNkDDuW8teVbRczS/bgQdUHuWR7KdCVsHJvsvW X-Google-Smtp-Source: AGHT+IE0sQfuin7/JVsoAE87oY5fS7SQLnr7XZ5bTFcFyFLA/EQRnCuiOON81YtnDRg9xulpU2U96wRlEAu4cZKUf3M= X-Received: by 2002:a17:907:bc7:b0:9e1:a5eb:8cb4 with SMTP id ez7-20020a1709070bc700b009e1a5eb8cb4mr12107194ejc.58.1701296529073; Wed, 29 Nov 2023 14:22:09 -0800 (PST) MIME-Version: 1.0 From: Chris Ochs Date: Wed, 29 Nov 2023 14:21:55 -0800 Message-ID: Subject: Non eal registered thread flow To: users@dpdk.org Content-Type: multipart/alternative; boundary="00000000000045fbae060b51f7c8" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --00000000000045fbae060b51f7c8 Content-Type: text/plain; charset="UTF-8" Trying to get a handle on the best way to integrate with my existing architecture. My main application is in Rust and it's a partitioned/batching flow. It's an end server. I basically send type erased streams between partitions using SPSC queues. Work scheduling is separate. Workers basically do work stealing of partitions. The important part is messaging is tied to partitions not threads. So what I think might work best here is I assign a partition per lcore. I already have a design where partitions can be designated as network partitions, and my regular workers can then ignore these partitions. With dpdk specific workers taking over. I designed the architecture for use with user space networking generally from the start. A partition in a networking flow consumes streams from other partitions like normal. In a dpdk flow what I think this looks like is for each stream call into C to transmit. Streams would be written mbuf aligned so I think this is just a single memcpy per stream into dpdk buffers. And then a single call to receive. Does anything stand out here as problematic? I read the known issues section and nothing there stood out as problematic. --00000000000045fbae060b51f7c8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Trying to get a handle on the best way to integrate with m= y existing architecture.=C2=A0

My main application is in= Rust and it's a partitioned/batching flow. It's an end server. I b= asically send type erased streams between partitions using SPSC queues. Wor= k scheduling is separate.=C2=A0 Workers basically do work stealing of parti= tions.=C2=A0 The important part is messaging is tied to partitions not thre= ads.

So what I think might work best here is I ass= ign a partition per lcore. I already have a design where partitions can be = designated as network partitions, and my regular workers can then ignore th= ese partitions.=C2=A0 With dpdk specific workers taking over.=C2=A0 I desig= ned the architecture for use with user space=C2=A0networking generally from= the start.

A partition in a networking flow consu= mes streams from other partitions like normal. In a dpdk flow what I think = this looks like is for each stream call into C to transmit.=C2=A0 Streams w= ould be written mbuf aligned so I think this is just a single memcpy per st= ream into dpdk buffers.=C2=A0 And then a single call to receive.
=
Does anything stand out here as problematic?=C2=A0 I read th= e known issues section and nothing there stood out as=C2=A0 problematic.=C2= =A0

--00000000000045fbae060b51f7c8--