From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B0550A00C3; Thu, 20 Jan 2022 11:21:27 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 496FC426DA; Thu, 20 Jan 2022 11:21:27 +0100 (CET) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by mails.dpdk.org (Postfix) with ESMTP id DAE0840042 for ; Thu, 20 Jan 2022 11:21:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1642674085; x=1674210085; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=ZMZh6xOVifcSlsLxvkMU9Omq9LvTlqQrqa4XfCq6frs=; b=Vne80p+Ytr2wEjc8cpHeOZl3cjPkYwQcG8J17ypbGvWG4XUZcyPoFZiY 9trpvDH0aoSyzlbU55eU2YbFXNsVUQpu9XhftQMFXTK7TwZ694JvqhDEo jkot4UPxmv84Kwb/Ru7x/X9EFUtsmdmjwU87bm/1IhnaWqXbTIoraH2sb AtiCktfCgBBUAeAmSo9vq9G5r6Z7sUOIo1Kt/ZLmx0tBqBLnMIjQjkkje HNJuAB5aFxIkbMfpZpgnOhYqakkejUp67W5832YYHLUmFsSALU/c22aVx Oy8dMTIFwyA/TD2YwCCe1eM6vaUpyou2WQ0cBwpIGrEl1GfpfzBeWm1QR A==; X-IronPort-AV: E=McAfee;i="6200,9189,10232"; a="244140819" X-IronPort-AV: E=Sophos;i="5.88,302,1635231600"; d="scan'208";a="244140819" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Jan 2022 02:21:23 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,302,1635231600"; d="scan'208";a="493376040" Received: from fmsmsx606.amr.corp.intel.com ([10.18.126.86]) by orsmga002.jf.intel.com with ESMTP; 20 Jan 2022 02:21:23 -0800 Received: from fmsmsx612.amr.corp.intel.com (10.18.126.92) by fmsmsx606.amr.corp.intel.com (10.18.126.86) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Thu, 20 Jan 2022 02:21:22 -0800 Received: from fmsmsx609.amr.corp.intel.com (10.18.126.89) by fmsmsx612.amr.corp.intel.com (10.18.126.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Thu, 20 Jan 2022 02:21:22 -0800 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx609.amr.corp.intel.com (10.18.126.89) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20 via Frontend Transport; Thu, 20 Jan 2022 02:21:22 -0800 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.168) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.20; Thu, 20 Jan 2022 02:21:22 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HYzBdXjF22iD1h+uEJMQDr/8r8BsRhyB+fxGbgHUZmyYEwVAs2TwSC3YbtybMjNdcEvn1YDYXgd1R7wBb8wLPFlP57XV3/ekXST0jGsIU3RRH6tnxgroQ/am9bL9J0wavuEx8lqTebtFmCZDJexLJpkZiTc+wuWYyunB8Z7O3+1wr2QAET/QAdZFYkLu4lM8c/ietGTUgSJ7Ezn+JD0cc5PKYDEnXdxpVLt5YCp28v6QDCJBodstW0dulSO+f0rlA1CaIIXl6oQog7NKkL8z5Fl/aibr0noEyDp4DdksEsoKPoAZ5QPym5mwT2W7kouzLauTUNJekQL1UgU/E8llkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ZMZh6xOVifcSlsLxvkMU9Omq9LvTlqQrqa4XfCq6frs=; b=mTtfCuV11m5hFuUcnEchg7gfCf8kATen7DZdXPFmN+RSToh7npqXp0DcjgmYGYMbscTqe+Fyv0Fx0bv6wkAQsOYoF8dO1wd7VmnCduk+X7jnni60r2IMR6T5dqHDt56v7QesBSti++A1mHOzZLPVpa8h57SJ1DmFFb1N4h/ifFYfL7bfsUrYh4xmK0ceCZEaEB3GZF+DxyKf30sk3oTA5WWJ8Bzwq/j6D4Qss62BbyCjdeI3aAkubAol5LogsOuqTvytv+ujxhhfgIbOW/lg7wgmBrdtztxc80/TJHaxcytAJCesiRSfABLfGdeyxCgkJ14zwlaUvRjdhC+aJJlFjw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none Received: from BN0PR11MB5712.namprd11.prod.outlook.com (2603:10b6:408:160::17) by BN6PR1101MB2258.namprd11.prod.outlook.com (2603:10b6:405:5b::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4909.8; Thu, 20 Jan 2022 10:21:15 +0000 Received: from BN0PR11MB5712.namprd11.prod.outlook.com ([fe80::343c:58e8:2190:8e7d]) by BN0PR11MB5712.namprd11.prod.outlook.com ([fe80::343c:58e8:2190:8e7d%6]) with mapi id 15.20.4888.014; Thu, 20 Jan 2022 10:21:15 +0000 From: "Van Haaren, Harry" To: Thomas Monjalon , "Randles, Ronan" CC: "dev@dpdk.org" , Jerin Jacob , "dev@dpdk.org" , "Richardson, Bruce" Subject: RE: [PATCH 05/12] gen: add raw packet data API and tests Thread-Topic: [PATCH 05/12] gen: add raw packet data API and tests Thread-Index: AQHX8PS2vuKotAJBk0+65GoyaSBuEKwzgAAAgAMOMTCAAFOYAIAERw+AgC9+qwCAAT/XAA== Date: Thu, 20 Jan 2022 10:21:15 +0000 Message-ID: References: <20211214141242.3383831-1-ronan.randles@intel.com> <6332947.G0QQBjFxQf@thomas> <5931992.YiXZdWvhHV@thomas> In-Reply-To: <5931992.YiXZdWvhHV@thomas> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.6.200.16 authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: b43dd098-36f0-496a-dafb-08d9dbfe971c x-ms-traffictypediagnostic: BN6PR1101MB2258:EE_ x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Cs3pbeO3fQj550Jvi5srSogcTh/UcY8L8iLYCkRAMTzO337DdO8uiBngjS9BAn9u4dbMi/mLfB22AV/H1iN+K6AQPN1JtsNs1iivjd3JWY4P+iIibbIlZdMMVRclSPEcNsNrWAPSezkFP/XrOJrQZVvINNPLwRh194DXMcQ46wfu1mpzenj24P/GHSzpQiIBJuPBmQ+wKlnx4dXnMJJYtOeL7LlKZS4lneNhhQWRb0CMeZFeBd7ufvjRgTsqD20q8J472Y/PphzKY39TYJYIFKaCT5FyIfJojLvsDqdfHU6hkB99RVCon1eu7KegpqVtyMxmm+aqARuZBQ5yVRKfUmiBlRtqzSkrfPH+FBbPL2X6qS6QobFiceCkES/5DUiQwM6DvOmWnXMbA6Gl1jSH3XzqVO2Qdmo1x9TmI9vlXyR7VLP66n4Ne4R8cyGDLGFInCylSvCCPtEbFj3M4gUmRC0kbZZVbVwZPhUqZ6oFOsPVaHfid4UgfGM2EuyiRYUfDqiVeoxbgzmQ+L04VHbdyW2vyzCxt2IjNM3Cog6uBWWWTi4opR9N44GPC9yBM9tjXJvKhPo5wo4gVuLgiYhgDrIdqwfU9O2FbHM7PnwCs+eUWVfF9ej+s595eunCzrqza/Tvqc2JCuIi2KYw2TTyJgpGXOo2F49FtrwIj5sDUwLpCrjoRNvDklsATS6gy/TPd2+IOIRmVgD+3XvyaYn2+AZpO+YSzBaAs1UQ7U7R7vWsRgPXFJcPQfKIgHdIyn2QBPJXQQ5TNLwrfvNdCdzAenRTl4RfJDe2+TsXNf4hIO1YUttmPsJfY0klPdEVGoDBn8pN3n7APEj6eQIFbAVnvw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN0PR11MB5712.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(52536014)(186003)(66446008)(66946007)(38070700005)(66574015)(107886003)(8936002)(55016003)(316002)(9686003)(66476007)(966005)(76116006)(66556008)(64756008)(82960400001)(86362001)(53546011)(6506007)(33656002)(26005)(38100700002)(122000001)(4326008)(7696005)(83380400001)(54906003)(6636002)(15188445003)(71200400001)(8676002)(2906002)(508600001)(110136005)(5660300002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?md6R8a+0/YVJ3Lj/nlBkAnD5nLqySVWcWXS3nEixVAAoXNjqOFuU6ZRjnTw5?= =?us-ascii?Q?gm7lEW1e1P+OfEfZr6IQroBRBO3DvCNUbElkwcVkyqVBQrcuraD7iDygKOfb?= =?us-ascii?Q?Onp6AlGFa2TRGKGEWFNjZkniOf+K7mviqiMLAjaA4AUEbG96x2rYja6WRei/?= =?us-ascii?Q?QoGv2cb0pQdftJDJOw8H4Cy+K+z2yax3tKnFMzLNwiNVH24qrehki9OIDe7T?= =?us-ascii?Q?wXQIG244axRyXSEHeCZC+KcCETxNviN+Qb6uQKG2DadQaPpOb6Mnv9MiC4Rk?= =?us-ascii?Q?hKFK3Az23erv8RnWCQ9/TRpSjcKK1DQQpErGLgl+j2NoYOiplxxqHmyoAU+S?= =?us-ascii?Q?drj4VwlVo1l7l05HGUzv4pl7loewRAyNJFWossBJUSeghKbhXF2VxkX0TJ7x?= =?us-ascii?Q?tJsM6itnZVTia7HvGW1zI2JuD603vZVW17l6gF743stNINz2j7gD3tvFQgdH?= =?us-ascii?Q?YrxIs/yX5Qb+fB1O1dOONi5/haD7HipiYqMJcGu79gHgxCMQhKrSJp7dga5x?= =?us-ascii?Q?P90d61Zv4c8wv3fJzk9iYPMjwQCkaHfgxGITvwpD3NkRp/xoTke5IdikBn2v?= =?us-ascii?Q?YVFWfM2T0/pHreYcoVdJCRqrVhqm6zH6B85DdnLUNmzCOm3d73j7tP4+rYxY?= =?us-ascii?Q?YuyxlLGs52ULW2d0++c1lAu6IFk0o9jJttgHq+XL4jmv9M6XL5yAs2px05Ed?= =?us-ascii?Q?m2tiXqpNa23OSEe/DDtsDxnkbHTH3nVvx5+Z4IBp45/vWQ8t5QOgT9H3wbWC?= =?us-ascii?Q?Bs4ISNDe9nARKKUN5xurhiHTzAN2BZY4ZOxnIVkVcTXw1RJnz3KYYITlilJl?= =?us-ascii?Q?H5Vf6r/4VNr8fn3Pn/ohRSntthIss7PZHHZ1RKFzS4SgCESG39t1aJebzXRq?= =?us-ascii?Q?U3L7tm5GWqmVdcernvvCdx9TsoHDPn7PU34E9fFF60s4ip1MeQ8e2Cf0WvxW?= =?us-ascii?Q?jx5xfKkQD9zOO5BWYHHcgLX1St711UJh1vTddHuzig+ndPhQIToh03viNlWZ?= =?us-ascii?Q?T0RKDgB8ny4wc9VGCgdXtcLjNanKL/m+1YE08K63XH1tHqhY4+vcQJcg9Rf3?= =?us-ascii?Q?87ZOdb4JrT376KMhmyt/K1iPqk5NdYHuzBGgcLj3bTY1Eh+eglRWWQ/35n/H?= =?us-ascii?Q?Seil3SwqA47z1OElYKUfvCSu7A8MEKEDmJpU9FWNnK5OdFPlW55v6qugQKoK?= =?us-ascii?Q?as3K2j5ahuU6ETCp+KhZTuAscs0dY/oomKqVZyTK5/FXoLcvyTWVrXifcSf9?= =?us-ascii?Q?7+ziPETuy01iPOZVUcLCtgpeGFoj3O1/Ppj0YNzCG+PjEnXTqn1D/smAXL16?= =?us-ascii?Q?99U9EHusTP2Sx/eX0eBSlVP67p9QjbEXGZber9EAATNqabcJRyBGx3bdJFIm?= =?us-ascii?Q?dkwfWqVNmAL2BAyPpT4H91SEQsNPuT2RyEUniLcScKNLp0s2dYUP1+knwZLA?= =?us-ascii?Q?4SG29bQDCsYOlTKOHRhMHpqHgw69n6U+lV+WpHS9mLA+bGmfqGe1T0NIENMw?= =?us-ascii?Q?kpaxwfbGIyM5pNo4euhKfD+5XRZwoCl3jwUcsBYS/Sw9lcxFBKt0jZpQY3Tp?= =?us-ascii?Q?PqJzwHB48jt4X0Z/J1/lgCaDQz5g5XTfGIrBQ6sztDhPJKaKobYr27o9K4Y7?= =?us-ascii?Q?j/qsXeEOJ4BJ81qT1twEhpGxbYeLrgFuQPBv5/Bc3N9nxm8L4lTVEXxBgkkK?= =?us-ascii?Q?sJU8wg=3D=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN0PR11MB5712.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: b43dd098-36f0-496a-dafb-08d9dbfe971c X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Jan 2022 10:21:15.1643 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ZizZJNS6bLajllxo+GhbSKgBHwTifCOeLommBbWqzwbCN06Fn5aPkh9UUxYOCCMylP3Z+kmjnoeRUAl7gtCxST1mNJGv6PeNdAl6k+OEsh4= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR1101MB2258 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Thomas Monjalon > Sent: Wednesday, January 19, 2022 2:56 PM > To: Randles, Ronan ; Van Haaren, Harry > > Cc: dev@dpdk.org; Jerin Jacob ; dev@dpdk.org; > Richardson, Bruce > Subject: Re: [PATCH 05/12] gen: add raw packet data API and tests >=20 > 20/12/2021 11:21, Van Haaren, Harry: > > From: Thomas Monjalon > > > 17/12/2021 12:40, Van Haaren, Harry: > > > > I could ramble on a bit more, but mostly diminishing returns I thin= k... > > > > I'll just use this email as a reply to Thomas' tweet; > > > > https://twitter.com/tmonjalo/status/1337313985662771201 > > > > > > My original question was to know available applications, > > > not integrating such application in the DPDK repository. > > > > > > I may me miss something obvious, > > > but I don't understand why trying to add a user app inside DPDK repo. > > > > There are likely a few points-of-view on this particular topic; and I'm= glad > > you mention it so we can discuss it clearly here. > > > > There are two main parts to this patchset, the first is a packet genera= tion > library, > > with an easy to use string-based syntax. The *library* is designed to b= e > extended in > > future to a range of "useful stuff" to do while generating packets. >=20 > The text syntax would be specific to this application > and not usable somewhere else, so it doesn't make sense as a lib. The text string format "Ether()/IP(src=3D1.2.3.4)/UDP()" is not application= specific, and the items being parsed can be easily added to; that's what I meant abov= e. > > The packet generation > > *application* should have minimal features, and focus on ease-of-use (a= s > suggested below). >=20 > It would be either a limited application, > or an ever-growing application. > If the latter, it should not be in the main DPDK repository in my opinion= . >=20 > By the way, I don't think it is the responsibility of DPDK to generate pa= ckets. > I would prefer having an application using the already known scapy > or a graphical interface like Ostinato. > There are tons of approach to define packets to send (pCraft is another o= ne). > DPDK should only manage the Tx part, and optionally Rx of forwarded packe= ts. Scapy did not meet the performance requirements for infrastructure testing = use-cases. The goal of gen lib is to provide a toolbox for high-speed "scapy-like" pac= ket generation, the above tools didn't seem like the right fit for my needs; hence Gen libr= ary. > > In order to test the DPDK code, we need a variety of unit tests, and a = sample- > application to show > > users how to use the library (as well as docs etc). For me, the interes= ting part is > that it is a small > > step from a simple sample-app just for testing to a minimal tool for hi= gh-rate > packet generation. > > > > I think many users of DPDK first install DPDK, then wish for a tool to = generate > high traffic rates > > to test DPDK, and end up with a usability problem; DPDK does not includ= e a > usable packet generator. >=20 > I don't see any usability problem in using an external well known tool. > Learning a new tool provided by DPDK *is* a usabilty difficulty. >=20 > > To highlight this point; our own DPDK Docs simply ignore the requiremen= t of > packet-generation to > > actually have packets processed by skeleton: > http://doc.dpdk.org/guides/sample_app_ug/skeleton.html > > Our "quick start" on the website uses PCAP vdevs (avoiding the problem) > https://core.dpdk.org/doc/quick-start/ > > Even searching the entire docs for "generate packet" doesn't give any > relevant/useful results: > > > http://doc.dpdk.org/guides/search.html?q=3Dgenerate+packet&check_keywords= =3D > yes&area=3Ddefault# > > > > Users could internet-search & find pktgen, moongen, trex, or similar to= ols. > These tools are fantastic for experienced > > developers such as devs on this mailing list - we should *NOT* replicat= e these > complex tools in DPDK itself. However, > > building any tool outside of DPDK repo requires more effort; another gi= t-clone, > another set of dependencies to install, > > perhaps another build-system to get used to. Particularly for people st= arting > out with DPDK (who are likely finding > > it difficult to learn the various hugepage/PCI-binding etc), this is ye= t another > problem to solve, or to give up. > > > > So my proposal is as follows; let us add a simple DPDK traffic generato= r to DPDK. > We can define its scope > > and its intended use, limiting the scope and capabilities. As before, I= do NOT > think it a good idea to build a > > complex and feature-rich packet generator. I do feel it useful to have = an easy- > to-use application in DPDK that > > is particularly designed for generating specific packets, at specific l= ine-rates, and > reports mpps returned. > > > > Thoughts on adding an small scope-limited application to DPDK enabling = ease- > of-packet-generation for new users? >=20 > So you want a simple packet generator for simple benchmarks? > And for complex benchmarks, we use another tool? In short, yes. Make simple things easy (gen lib in DPDK itself), and make d= ifficult things possible (by using TRex, Ostinato, Warp17, MoonGen, Pktgen, pCraft, Scapy, = or other tools). But to summarize, it seems there are multiple questions/concerns around mer= ging the gen library into DPDK. That's OK, this is not a critical feature to upstrea= m, I thought it might be useful to the wider community. As some rework was done from the V1 to a V2, adding some protocols and gene= rally advancing the library, Ronan and I will post the V2 to list to make it publ= icly available. We can mark the V2 as "not applicable" in patchwork once sent. I will likely continue to use the Gen library for my own packet-generation = & testing needs. If in future somebody is interested in Gen library send me an email! Regards, -Harry