From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A262F41CB6; Thu, 16 Feb 2023 22:00:30 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4452440FDF; Thu, 16 Feb 2023 22:00:30 +0100 (CET) Received: from sender11-of-o51.zoho.eu (sender11-of-o51.zoho.eu [31.186.226.237]) by mails.dpdk.org (Postfix) with ESMTP id 8A7DA40E25 for ; Thu, 16 Feb 2023 22:00:28 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; t=1676581227; cv=none; d=zohomail.eu; s=zohoarc; b=ANzrJzIW9gvR9N/1muSZe8JjtqUh9CdMKvdeqE9Z3MZdRihugFlCDHQkMfA7rZM77qE+WQ3qL6thX7dHXH8YMgXrQXLB0sWoF8pObsE0m5Kl5cZHQv4vR9St2azC2hyUnjR1ByVFmY01R+JVlo03sXpkkAe3ptVfk/Iyax+DkMM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.eu; s=zohoarc; t=1676581227; h=Content-Type:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; bh=7VUk7Khteq1b/e3gC4uBVE8U0GfmWrVfKMAeLSWnY10=; b=MZZxScif1ohF+GXAIRIU1HE5Ano85TUuzK0wv+QxX+uL8oad+JcSHlyLp/BPilUaJnujm33g5Om5VJtBnjsr9TpZmVb8lnoK9aV28o4+CoPiUNlhVnpnD1eR7qHLs2Sce9aNUnaB6LJw4H4opWWKTlQmIUO1Oc1Gx8TJKT/QEjg= ARC-Authentication-Results: i=1; mx.zohomail.eu; spf=pass smtp.mailfrom=liangma@liangbit.com; dmarc=pass header.from= Received: from C02GF04TMD6V (ec2-54-93-135-31.eu-central-1.compute.amazonaws.com [54.93.135.31]) by mx.zoho.eu with SMTPS id 1676581225066535.8767027086285; Thu, 16 Feb 2023 22:00:25 +0100 (CET) Date: Thu, 16 Feb 2023 21:00:22 +0000 From: Liang Ma To: "Van Haaren, Harry" Cc: Sam Kirubakaran , "dev@dpdk.org" Subject: Re: TCP stack support on DPDK Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ZohoMailClient: External X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Thu, Feb 16, 2023 at 03:01:28PM +0000, Van Haaren, Harry wrote: > > -----Original Message----- > > From: Liang Ma > > Sent: Thursday, February 16, 2023 2:40 PM > > To: Sam Kirubakaran > > Cc: dev@dpdk.org > > Subject: Re: TCP stack support on DPDK > > > > On Tue, Jan 24, 2023 at 01:24:38PM +0530, Sam Kirubakaran wrote: > > > Hi Liang, > > > > > > Thank you so much for your response. > > > I would like to convert our in-house tool into a high performance traffic > > > generator and it is written in Python. > > > Is it possible to use any of the user space TCP stack by compiling into > > > shared libraries and access it via Python using ctypes? > > > Could you please let me know any viable solution to make use of user space > > > TCP stack compatible with Python? > > Liang replied: > > sorry, I have no idea bout the python bind with DPDK, but there are some > > Lua bind with DPDK. Like Moonwire. > > Moongen: https://github.com/emmericp/MoonGen and its "library form" > Libmoon may be of interest yes! (https://github.com/libmoon/libmoon) > > Note that LUA has certain properties that are nice for packet processing (soft-realtime capable). > Python unfortunately does not have the same properties by default (but likely can be configured/used/extended to mitigate impact). > This is only relevant if you want to generate packets "on the fly" with very low jitter (potentially caused by GarbageCollectors from Python runtimes). +1 > Looking at the bigger picture: > 1) What are your requirements around performance for generating traffic? > 2) And what are the reasons that existing projects are not sufficient? > Trex is very configurable for many things: https://github.com/cisco-system-traffic-generator > > As you likely know, writing a good reliable traffic generation tool is a lot of work, > and re-using existing work is likely a good idea. There is a list of DPDK users/libraries etc on the > official DPDK site here; https://www.dpdk.org/ecosystem/ Some of these libraries might > provide a good starting point for your needs? > > My experience when developing a simple/minimal traffic generator for DPDK was that although > It might seem that generating of packets can be done in Python/other-interpreted-language, it > causes performance issues in the end. Imagine generating 1000 flows, the checksum of each > packet is going to be different. That leaves two options: > 1) Generate all packets in advance, and "replay" them from memory (memory intensive, and no other customization potential as it corrupts checksums!) > 2) Calculate checksums on the fly in C or other high performance language: the complexity of checksum calculations, offsets etc must be present in high-perf language, removes benefit of using Python/high-level language for "quick" packet generation. > > The results of the Proof-of-Concept that Ronan Randles & I done is here, called TGen. I have > Presented my experience of "handmade" traffic generators as part of DPDK userspace: > - Video: https://www.youtube.com/watch?v=Djcjq59H1uo > - Patches: http://patches.dpdk.org/project/dpdk/cover/20220121103122.2926856-1-ronan.randles@intel.com/ > > Hope the above information is of use! Regards, -Harry >