From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D378C43407 for ; Wed, 29 Nov 2023 23:51:00 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5C2BC402E6; Wed, 29 Nov 2023 23:51:00 +0100 (CET) Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by mails.dpdk.org (Postfix) with ESMTP id A1628402E1 for ; Wed, 29 Nov 2023 23:50:58 +0100 (CET) Received: by mail-pj1-f53.google.com with SMTP id 98e67ed59e1d1-285f46e1cd4so378616a91.1 for ; Wed, 29 Nov 2023 14:50:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1701298258; x=1701903058; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=mwYmJuVOONy4bd5SZEEcI7g8aeIDKZSdC4B4kWUH5TY=; b=P7wtw8WCPgx8b2e2gOkFIxFShm4eTnQr8P6g4IRfMPUVt3oxDqLHq2nnIRbGxueCJ/ rqUe14bOMpfQOAQF37M4d8w+3JRGZUoarRZlESb8YdFoyqeTB8noaNtefGcdfrizCDL5 a9P0Ba+qeMecvM4tJ4YhOt806Yf56ChYbJR4OzDXoYTwS7M0OIHOV7D8zD4YEFvDZYeU 2/tdunIYv9oyfXqvcYizs1kR5gAvUnsNVwRQHHjjCYvuqeJC7W1M360efRSZQRCDfNXv wHuIoTBl+gq8O1W/UbIOVQo553UiwiprIZD1d3TtHT9QnXiDsvNtu7uuGQ1Zm3PxopNy rJNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701298258; x=1701903058; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mwYmJuVOONy4bd5SZEEcI7g8aeIDKZSdC4B4kWUH5TY=; b=W8OdN7pSDb1PrMLJdnx0tq6J5DUWdPIo/+aM5LlujpFNPcBKc440XEQ/EqCTsWXNem x4bplGC13bUtPnNBito48F6Wk1p9XwoNCoqBkV/07p4XMlGdmPHOiFE1NlsU6Bc10LEn qdYYadUGhKR3dQRs9Odq43s/9rZQOgm7UtwysYvYqq1SXZGkM2y5f9HQpXNZ8aEPwjZ/ 1XQ87dgKtyacEBQZVCrqLf8kf1Wn5fn8StwIs0zxMG0q9ohxGwZlT7n82OyCXkK/kcmi nRIv4xljiCpqrwDgNO0zTi4BfTZCNpq4fATPZqdbCwN8k+tYgVmk2Cy8ayiljTb7lM9A Dc9g== X-Gm-Message-State: AOJu0YyZ56MAW8HnY6O/U8GbJ5+wvaIDEVlvyWdCdlB4PHn/RmA6Igr7 WWRzA32iCP8jLvYT5SamgQh+qw== X-Google-Smtp-Source: AGHT+IGNQmxJti9mnsrQqrSe/LGbSiNTPa1TfbbJiAUQ8JPIy9+kdgdpnCTRmaWaa3LacdeMUTXatg== X-Received: by 2002:a17:90b:3a8c:b0:285:d720:b568 with SMTP id om12-20020a17090b3a8c00b00285d720b568mr11337810pjb.27.1701298257617; Wed, 29 Nov 2023 14:50:57 -0800 (PST) Received: from hermes.local (204-195-123-141.wavecable.com. [204.195.123.141]) by smtp.gmail.com with ESMTPSA id fs13-20020a17090af28d00b00280202c092fsm1901748pjb.33.2023.11.29.14.50.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Nov 2023 14:50:57 -0800 (PST) Date: Wed, 29 Nov 2023 14:50:55 -0800 From: Stephen Hemminger To: Chris Ochs Cc: users@dpdk.org Subject: Re: Non eal registered thread flow Message-ID: <20231129145055.4c25d5e3@hermes.local> In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org On Wed, 29 Nov 2023 14:21:55 -0800 Chris Ochs wrote: > Trying to get a handle on the best way to integrate with my existing > architecture. > > My main application is in Rust and it's a partitioned/batching flow. It's > an end server. I basically send type erased streams between partitions > using SPSC queues. Work scheduling is separate. Workers basically do work > stealing of partitions. The important part is messaging is tied to > partitions not threads. > > So what I think might work best here is I assign a partition per lcore. I > already have a design where partitions can be designated as network > partitions, and my regular workers can then ignore these partitions. With > dpdk specific workers taking over. I designed the architecture for use > with user space networking generally from the start. > > A partition in a networking flow consumes streams from other partitions > like normal. In a dpdk flow what I think this looks like is for each stream > call into C to transmit. Streams would be written mbuf aligned so I think > this is just a single memcpy per stream into dpdk buffers. And then a > single call to receive. > > Does anything stand out here as problematic? I read the known issues > section and nothing there stood out as problematic. Are your lcore's pinned and isolated? Is your API per packet or batch? Are these DPDK ring buffers or some other queuing mechanism?