From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6F3A141CB3; Thu, 16 Feb 2023 16:01:43 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 525E940EE3; Thu, 16 Feb 2023 16:01:43 +0100 (CET) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 32D4E40E03 for ; Thu, 16 Feb 2023 16:01:41 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1676559701; x=1708095701; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=iwgMtuPjX5d4yPZZgIoRuQ1bmIslHe8DBtctZp/PrjY=; b=XA2v29VxpAlWLCnhIAqQvO3owjR6uGfEMJgWYRnEGfSZFjnz2/1yeFih bl+yAwictJ73MRdlBEkLHMbZ1ojZOJGRfmXcqAIq2wFHZ1Z7EOCsNcaTV 71q3Cqpf2dOegUagu+nxctRLl5BLwxoW2GCk2oqAn8m3lM7Qr31X3WJpM X80ltPiEHlcb6k6opebW3jDAKYnQObXKpUNR5yltLhpYryOHF+y7TT/EM XO8vf5JcZgryShjnXpM9O/A4DBFfmFUQmSas8G4vdB5FEb4OxnYyscgK9 wT6Z4pxtHuIjtOSEs8ZTMt+H2Leoetjb2CTG29Xl5WandmSK9Bb1kliVR Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10623"; a="331731858" X-IronPort-AV: E=Sophos;i="5.97,302,1669104000"; d="scan'208";a="331731858" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Feb 2023 07:01:32 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10623"; a="672176998" X-IronPort-AV: E=Sophos;i="5.97,302,1669104000"; d="scan'208";a="672176998" Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83]) by fmsmga007.fm.intel.com with ESMTP; 16 Feb 2023 07:01:31 -0800 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16; Thu, 16 Feb 2023 07:01:31 -0800 Received: from fmsmsx603.amr.corp.intel.com (10.18.126.83) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16; Thu, 16 Feb 2023 07:01:31 -0800 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16 via Frontend Transport; Thu, 16 Feb 2023 07:01:31 -0800 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.169) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.16; Thu, 16 Feb 2023 07:01:30 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Rp1vfeSHBWK0CI7LcreFb69WrL67X+7sDJL+ofkAtRb/vjAf2EWHQFSFLBamCBCFPqOaf9Xl3G6SZzbfbWJ9VTp2+hCWfexoGCGjtbxLIXxeULtxdUHn1unigeQXnf8Yx442IZetCLwws/Nj3SppzGMai9Cmi2+zptt8J9geYmNkZbwLlkC+IDR6pJ2JpiU+Ch5hdk1DNX8NqAb1F0jPXicTckFx19lSs3R76ArTphd+5kfSe0VKsBfAAtvIBLL1D0IFpPflcwdEC21HEcSBJl4QzRx6eKA6MOKym+BY3P27UP5NFWPT9rlNaKvSWtSmG0sg9ysdDBrimirMJQFlvA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=79+rYAK1l3JyL8oWW6Fd6BQwaNDkxGxPxzdWlhc9/RI=; b=kQZlA+Bb3RUSbq5jjse92Qk+eCcdjk57RtcNK1oySgZ8HBwsGZe+dvTjn4CSktO/CRtEoUGRCEXeGddzqTZwRCqpYtO8sSmfY7cuzIfGPACecJTzLt1Tg2RAg1ODYdG98q2BPD8b7T5Vbr5z0g/33LljskI4kN09NCdJMrSOguJOmvv5PgQbH0qw+YdoxoWKyoPAQka3EZFraoBgwEhc/T2fjeucaq4DViHhnmpc8wLwt4EwbPipphvYx0OdHXQDUXTIcx2OsBOreccv6YMDs/4X/mtmWoXUAaNQVPYIf3pe6UeQ+drUVjhWNMN64LfyMCpsHmR0b5iKrxj3ViDVIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from BN0PR11MB5712.namprd11.prod.outlook.com (2603:10b6:408:160::17) by BN9PR11MB5306.namprd11.prod.outlook.com (2603:10b6:408:137::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6111.13; Thu, 16 Feb 2023 15:01:28 +0000 Received: from BN0PR11MB5712.namprd11.prod.outlook.com ([fe80::f912:bad2:e618:84c1]) by BN0PR11MB5712.namprd11.prod.outlook.com ([fe80::f912:bad2:e618:84c1%8]) with mapi id 15.20.6111.013; Thu, 16 Feb 2023 15:01:28 +0000 From: "Van Haaren, Harry" To: Liang Ma , Sam Kirubakaran CC: "dev@dpdk.org" Subject: RE: TCP stack support on DPDK Thread-Topic: TCP stack support on DPDK Thread-Index: AQHZLMEBZgeX2YjRlUKo0UyytU+X1q6rzqUAgAFqDwCAJJbRAIAAAYCw Date: Thu, 16 Feb 2023 15:01:28 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: BN0PR11MB5712:EE_|BN9PR11MB5306:EE_ x-ms-office365-filtering-correlation-id: e94b0a90-256e-4d41-8499-08db102eae7f x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: oqMwEP8zHa+U9MrhYKBMMmBBGBCHCa1g4rW3HhPnd18ABQ0iARK/YSp7Pgpq1+8SdwQJn9IpfDgWaxghscZSUKn0GE4GbkspsUS0Iw7ezo6vOaRNgHsMqJ0TMz9+qwMBEK7DHlrH5YDmxBNLrRfdzosbt0IAR10La3ggZf6rxMzAPD+wo0WjP4mmDdSwfbD3SFZ31lh7trda7Z0xre0LUaDK1kFDG+IDfEJRekxXyJxiXkFZUmzgJ7F4uxmC1imnz5xQUNIQrjZQfZ5cBcorbDLBGrzGzCguH8QJFkUc/9k229JVYuW4kPod+38zYRspB0id02ya9PSIw6Yx4szKxNTL83JFWDgeARaKgSrBUZ0DMlLXWLQsV2pOE+Wx23p7G3s8K0M96YNiBnKFaQvArAFOJlCGHW2yQmSNpOOL7He7WH9FS/yq6Cy+Fab1FkaOtEY3hi7hOazdbZnSor/q3hQOyePcMzOHUL34HSIG4rY4cZ85zPCnaZ1yFc9eVaIAPQJt0oSzX87HZLd0ejQKTE0bFQesiu1dfQdxUYWeDVAErpZyf0HELrH/nRMwcFqRmy90jOtlSaShwwODcvFr0pC7IudVidbX5X9nf9Ka8YXiSdT9TEp9OOVq7TCLPhA2wR/iL3BNPmZyAHI5Ql9NsQZGRQdI9/9Jf33zKW7IxRV86Bas+3Pa3nyv3UPyMiLa690yp3fHXWoL+yvXscaNV2PnP56wLplTsKQYASHiPqM= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN0PR11MB5712.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230025)(366004)(396003)(346002)(136003)(376002)(39860400002)(451199018)(110136005)(86362001)(55016003)(316002)(52536014)(8936002)(5660300002)(478600001)(38070700005)(33656002)(66556008)(41300700001)(8676002)(4326008)(64756008)(66476007)(66446008)(66946007)(76116006)(9686003)(186003)(26005)(53546011)(6506007)(2906002)(966005)(71200400001)(83380400001)(38100700002)(122000001)(82960400001)(7696005); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?zivpqiZWVpRmsCitMakJXQsdZwvX7qaKmx/rlJhDQTtLnEZ1CgOpE+iI2hhx?= =?us-ascii?Q?6QQmCa9/mGTzAF1e+xfKVRUJ3KVbvJy+GSBRV1S0fGTBk1cBvBZ7DjOEkCSH?= =?us-ascii?Q?DaxAfpTun3jgf6rIZ28SfbKhb1Km2qUTTmN7TagsZZSJRnsySjb/uTVHXySc?= =?us-ascii?Q?WC6HmMT/4jTSFl80oy30V5H2ID3mK5dG5bG3Zvo76/A0pyxoqrUyrZ/0UWa5?= =?us-ascii?Q?ledGgnHvCBr9aBqanv0zdxIobmZJC2d2jGLJbUl2OvoFwJ+IegyQxEPShyRA?= =?us-ascii?Q?Ihu/vqgb7RJqLnIl3kSSN6a99B52mWh40KjLSwkWyVE6HO3uHXQl8uVfifBV?= =?us-ascii?Q?yhdgTx6B2VM+rxUbQuHS10DXVZM+FCz+epXwIL5KBkQQswUvX5JuUnGhrMcF?= =?us-ascii?Q?mvlDbewBX5ZrstNDUE7+OTZDdggyQC6pG+duiWl3gGG1DyBoaAYk0JyaENt5?= =?us-ascii?Q?Dhr2m3otRp0DJQtZ+91TIN+SAK9mYqnJcwmQdK7noekmkMVpz0ghja/TVOEA?= =?us-ascii?Q?z474ycf/gHotoDj8o4GMYxLOAYaWacAZsRCTcchv/MTdJ9a84A/whrquS0VO?= =?us-ascii?Q?ek7miISIm/cNoky9tqeimGIM7hihmNugz3/vmy9n8WoF2IRScxe4ycLQpoAa?= =?us-ascii?Q?MSbghnSYxbFyL8btkBsJSlopnSNWqxESluSZFbEzs+jpbrOiVGaamsQS+Q6s?= =?us-ascii?Q?ssRZTBNGJhSPmjac1RpTwYo4EeDQtkgrZz2kOcYDF/7cMAAerDyEPR0RsuKN?= =?us-ascii?Q?wxxOrNqxFvsDzRWUsqgXJcmVFJ6gbzIXMeCCM8yOdCVxqdYsiJi9/KrtMz5i?= =?us-ascii?Q?nuvzWM/q5rvIN+PyxtRv173UqC9RcGTMXcRt6y9tUbwqjNTrtq3aRR6B62nE?= =?us-ascii?Q?8F2Ek7ht1SamT3WpTvQXB1AOFDBLaLMNrR+Ko7D+d1Zcl12TSk04LnLrSLC2?= =?us-ascii?Q?L942IzPBs895bexWN5p/w4/HWU6m3+zT7tf/Q2cyas4+ysIMJCbbFUuzepso?= =?us-ascii?Q?O5yOqerFn2Iks2aOGVzaGE791R7BWzzs2XgtTR4K6rTzn/CSZ5a6cHTDu4Sw?= =?us-ascii?Q?Fhb6EaxqS/pxpr8BIluCJK8vLn5C2Pa7DWrjXBOdH5UdXYUAi/ml22B3zI+O?= =?us-ascii?Q?oSTL7pXfwhDciPtCGr7+X/6+jhoPPYpcyqpozqOS0wDyT0nusKP8J9mRM2kr?= =?us-ascii?Q?0JrKnU+WI1RSJgN8cHu+2Y1FjzEmmd7EHow1nBWbSZRfhPNjwBakK9QsBVp9?= =?us-ascii?Q?Gv7m/0wMMixChgfh9y0uQ5eFKC0NdwiL1lrmuSZNdbns39Vl4nudLJ562Au/?= =?us-ascii?Q?GXAiA4wyiSMvRCdFgmshiVmXUp6HRCcoVb8x5ZUdzv2jh7vOTsVGhVSgZISA?= =?us-ascii?Q?TDScR/WakpwijzINIaSy/grxWy2qWPSNHNZxje2u5rYmhjOxxmYL8+X76mhR?= =?us-ascii?Q?JrxYDaRZXKJYu5lqYs1Dd19aUOJSh8njXcFNbdAvu12hdeSyJvxZ0EvaG1+C?= =?us-ascii?Q?RBAcv+acEVcv4qurybqt+cjAInGYqdeBM60G425Vg8goEdhwD4AwY8xJlVls?= =?us-ascii?Q?QVJ644XQo6hLJomCZLG9jFujEMu7e8uL3SWSbTBn1y8jnsUGHts6+De+07VR?= =?us-ascii?Q?4Q=3D=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN0PR11MB5712.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: e94b0a90-256e-4d41-8499-08db102eae7f X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Feb 2023 15:01:28.4721 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: HjX0Z7gSTejDQsRDSBuNS75QoibPSHayccC0Zy710i10NMwHhp+BUBZj//CGBnPf5w+WDR2vC4/TO7I2x/xJfCT5qImYDJOeRU3uvbpIp7M= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR11MB5306 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Liang Ma > Sent: Thursday, February 16, 2023 2:40 PM > To: Sam Kirubakaran > Cc: dev@dpdk.org > Subject: Re: TCP stack support on DPDK >=20 > On Tue, Jan 24, 2023 at 01:24:38PM +0530, Sam Kirubakaran wrote: > > Hi Liang, > > > > Thank you so much for your response. > > I would like to convert our in-house tool into a high performance traff= ic > > generator and it is written in Python. > > Is it possible to use any of the user space TCP stack by compiling into > > shared libraries and access it via Python using ctypes? > > Could you please let me know any viable solution to make use of user sp= ace > > TCP stack compatible with Python? Liang replied: > sorry, I have no idea bout the python bind with DPDK, but there are some > Lua bind with DPDK. Like Moonwire. Moongen: https://github.com/emmericp/MoonGen and its "library form" Libmoon may be of interest yes! (https://github.com/libmoon/libmoon) Note that LUA has certain properties that are nice for packet processing (s= oft-realtime capable). Python unfortunately does not have the same properties by default (but like= ly can be configured/used/extended to mitigate impact). This is only relevant if you want to generate packets "on the fly" with ver= y low jitter (potentially caused by GarbageCollectors from Python runtimes)= . Looking at the bigger picture: 1) What are your requirements around performance for generating traffic? 2) And what are the reasons that existing projects are not sufficient? Trex is very configurable for many things: https://github.com/cisco-system-= traffic-generator As you likely know, writing a good reliable traffic generation tool is a lo= t of work, and re-using existing work is likely a good idea. There is a list of DPDK u= sers/libraries etc on the official DPDK site here; https://www.dpdk.org/ecosystem/ Some of these libr= aries might provide a good starting point for your needs? My experience when developing a simple/minimal traffic generator for DPDK w= as that although It might seem that generating of packets can be done in Python/other-interp= reted-language, it causes performance issues in the end. Imagine generating 1000 flows, the ch= ecksum of each packet is going to be different. That leaves two options: 1) Generate all packets in advance, and "replay" them from memory (memory i= ntensive, and no other customization potential as it corrupts checksums!) 2) Calculate checksums on the fly in C or other high performance language: = the complexity of checksum calculations, offsets etc must be present in hig= h-perf language, removes benefit of using Python/high-level language for "q= uick" packet generation. The results of the Proof-of-Concept that Ronan Randles & I done is here, ca= lled TGen. I have Presented my experience of "handmade" traffic generators as part of DPDK us= erspace: - Video: https://www.youtube.com/watch?v=3DDjcjq59H1uo - Patches: http://patches.dpdk.org/project/dpdk/cover/20220121103122.29268= 56-1-ronan.randles@intel.com/ Hope the above information is of use! Regards, -Harry