From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 97D7543795; Tue, 26 Dec 2023 08:32:08 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 901804029F; Tue, 26 Dec 2023 08:32:08 +0100 (CET) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by mails.dpdk.org (Postfix) with ESMTP id 0E9A640298; Tue, 26 Dec 2023 08:31:55 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AWkbKgQcUCCjB4B+7mtl93b6Y9pdeA0JsLU7ZR8dSqDL0Bpm1nFpIQ/rakDUn7EHXs7+b51jwG2aHDoUdhnG3P/tuwTpy52knWDJQwRPLgRprpa8En6RWHrtZC2DolP1+O+Gv18l6K6LefKrDq0nleSRnInkgzB4h9Yq109c8SqI58rJtpA9UQvOsOGWmYEOIkjC/fBLkrz91vuSuzRxiUvQRUjt6oCQhwcqQecChsB0ZhH/SNvjEAJOLO4QTJtu7HkThFWaxtKZKi2GpmOlNs9LU+K1WHwsBG0InYDINzcAQOAsy3X/ungiNWzZDr1MncwX+y6qOVKpdMv3cxg5Mw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NOw+oCDR1MYLfA12PFiNN90mW4cP6LW1GP/hmlPxScI=; b=jrxxbzkkTxpVjWT4XTFB+FphuD7454IVBupLF2jgn2wJc+4BtEKHc6JmYeEN00W/Z8tC6qmsdr9DJgviKFJLf/3NP2SpBU7+1iMiCxJEpOIRMojru7TKEJjXhoKUY/VTQK9nSinQDhzL6RMtixkF8V4GcVTZxmDGh0nKJTvUVH/u1fJ7W7AGPfPzLIqBgWvGLuf5eIptXY6wRlcqoTZqm+7e+Gy0U2OhDoCGI/vrroZT1Nd4GiWXD8X+L+5mRZ6b/VoofWRq1qGrBa7TjwMC7pJZCdH3UupJuXGE09CCp/It49fuWZhCEVwOXNTTo3HooOlK6P9w4ts9tPGaf+0KDg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NOw+oCDR1MYLfA12PFiNN90mW4cP6LW1GP/hmlPxScI=; b=H1M904txBwtbaoF+ZfpBZyQydbTpanxhnAV2KoOg91TtTKgEVAlZ5oBi0layWcNzPCEvIT6aSEYV85qWlkBveV09UfP8R24XO8mpWZGObGnJhzthxOt6UKgfM1R+CeVa7H7boiVcm5YHdfp1CLQao2L/lGVbOgKCUp95aLXGVM4Z0V8tSAJjuGi/TdnHBHOYSQayDmaLJhZ0hwq92i3OkpYnXYRULikF7GDvvuogYdyxcr7IJC0eUBYpS+4dU+uGrEwdkM9YTOfsQElfWq79y/gEGfyuifHOcq8krjBc7cHJBPDAAgkmFuvR8v92Qdb4cXlqRNImquK0nSJJPtIO0w== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from IA1PR12MB6332.namprd12.prod.outlook.com (2603:10b6:208:3e2::13) by DM6PR12MB4436.namprd12.prod.outlook.com (2603:10b6:5:2a3::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7113.27; Tue, 26 Dec 2023 07:31:52 +0000 Received: from IA1PR12MB6332.namprd12.prod.outlook.com ([fe80::43ed:468d:3105:4071]) by IA1PR12MB6332.namprd12.prod.outlook.com ([fe80::43ed:468d:3105:4071%4]) with mapi id 15.20.7113.027; Tue, 26 Dec 2023 07:31:52 +0000 Date: Tue, 26 Dec 2023 09:31:46 +0200 (IST) From: "Etelson, Gregory" To: Thomas Monjalon , =?ISO-8859-15?Q?Juraj_Linke=A8?= , Honnappa Nagarahalli , Paul Szczepanek , Luca Vizzarro , Yoan Picchi , Jeremy Spewock , Gregory Etelson , Patrick Robb , "ci@dpdk.org" , dev@dpdk.org Subject: DTS testpmd and SCAPY integration Message-ID: <2a287ee7-cda4-f2ab-a4e6-a47021f8573f@nvidia.com> Content-Type: text/plain; format=flowed; charset=US-ASCII X-ClientProxiedBy: TLZP290CA0006.ISRP290.PROD.OUTLOOK.COM (2603:1096:950:9::16) To IA1PR12MB6332.namprd12.prod.outlook.com (2603:10b6:208:3e2::13) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: IA1PR12MB6332:EE_|DM6PR12MB4436:EE_ X-MS-Office365-Filtering-Correlation-Id: efbf29e5-145e-4e2f-c0e9-08dc05e4bab6 X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: XNHwRjDF/KYWZAQH5Gi6vnBhGQpYTyT+POY4Wq7OgEOSbHD8d8lnIPqoGlOMgDJutxVlFaGmLrPKzlKBAow5dEsZrLn28d9sX7LsBOatA/2UAzIDWagjTLDVe0OnD+ueb6WiGMdA5iR1WEJf2rT/z7yr4hBwWQqjwpPHjDaIM38gD+ZVLLqBQwkfbNX900H6zWeCMKz9/ZBtEUlCu+EdQQNDfbtK3wkj9EqcJEVIEJtgZaUH9jXa3/7+dfNtcgy+OeMmxeAA69MWeCkuwi+sEpKV2mAssS/pm6XrjYbuyOSdxQZzr1tfV3JXrPEn7dvf3v3U2rjgXqFCJUok4SuF+U/+yND3wU/UElo14ed1Cuu7q2nh7HZXGULRjHIFWGHEHXl4e3yQ6Ir2SJXO0YAZNPaWnp1KaDVnOw1dM/o/FhC0+4d4f9VeZNzAizczx0nlTmsUOZoW9I/ZNIauW+7r/l5s1QEO5F1Tt1ssvoZbnksQT385xVLUfo9YAV3UEwLMnvkWe5Cu3G8GGf1Hemw5twq99vSgFZeOYUzupAssaIcfe/Y9VlyzdGhwRltiBD14Po6C6h6pqT8SJrNmx2qdj1pKVRRmN/6ocfP0NNBJJSgkXm8ymwc6hd2V7LKDtTlCoA5FDTiOyBTMuesfDT9drA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:IA1PR12MB6332.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(396003)(136003)(366004)(39860400002)(376002)(346002)(230922051799003)(186009)(1800799012)(451199024)(64100799003)(26005)(6512007)(2616005)(8676002)(31686004)(110136005)(921011)(66946007)(316002)(66476007)(66556008)(83380400001)(6666004)(6506007)(38100700002)(31696002)(86362001)(8936002)(478600001)(6486002)(5660300002)(7416002)(2906002)(36756003)(41300700001)(45980500001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?BwN5eqgU3813Ee/KKCtS8fRLXopc9Sp3PWcLnil04I2b5NhU4zkC4S58FRR0?= =?us-ascii?Q?bizZKOIfsTqERzYSnVzTFRG2pFas8a1BN6rb8LzonRe2txvhgodj+maHX2jL?= =?us-ascii?Q?kTB7HByNkpa/8TF470JxOovnBal5CnJce1PGRI5bPt1vUSooGeLS1OCqQ7bZ?= =?us-ascii?Q?nPRYq1U452QmEQTmJ0/YdKlmnCbbW3EcZo8Zag5jAITSmeZR1uMYvc8C0OXf?= =?us-ascii?Q?SRCRENaOoQfI1g4ttMGqUZrQaF0X6QwP2gkmxTN1sY1r2MUWwUA8PIcqfRFu?= =?us-ascii?Q?pFFWS/kb7W3WWV/y5Q13B7Y1mPLKlm7TL2OKG/61vnMnAlNPIR6W2TIQ//2W?= =?us-ascii?Q?rbcjN07kGDq3O8vnP4Tu2TJMueb19MDa5birXcZwF5p/k4VECXkbH4kgKXSp?= =?us-ascii?Q?Z5hkHNKuoVZiTy7sCN7+k7MpX4gCykWlaaWZKxYpjkSeJZKtndZXpi9oLCyI?= =?us-ascii?Q?h3i4f2oFnLECc1u2+xL8Vgj4mFU1W89BQjFR2VyQdcjEuWm3u4quRSU1oHhK?= =?us-ascii?Q?XhqkqoA9RN3ZlmYaIiaPaaImmwLJhW0RQl82brBu+w6ERUVSc8hYu93kLiZY?= =?us-ascii?Q?vS1pPoTxSDkDbqnRe/mBzrtUSgT0GbVdMsEcZoTosYDV+mdxejmbkkqThjxt?= =?us-ascii?Q?/mhtTF4pf/ff67oRcMkp/37F6i48ZBBqsdvS494B04Vx5e20ipUgWwbnsLm0?= =?us-ascii?Q?0mJKs+A8n91sC3yE4oQMroTPncK3UgjJuJD+kj+C+2sAa3mDKUQAG58BWQqj?= =?us-ascii?Q?jSm63esUTZEzljyWCJh61puIrU/1gTRuZedegHoasqh0WJ03X32wLIrRmTO0?= =?us-ascii?Q?PObFn+yzAHltvVpk1x7uo/BxU+d/zhVIe0Zg15DxqdzaJkGpJsg9vS1cPTLz?= =?us-ascii?Q?bcUtz4BkMfzERMKv9trz2NThXHaHeJ4o9H13IqrO0tjH8kn5k3zHUtJ3h7cr?= =?us-ascii?Q?AtvBf3u4cO3V5mKXztQOLFFCVeuFjoOMYN8tX5hJLgEn6FS0WzjPCCrBRgL3?= =?us-ascii?Q?UrNn7+n5zWG2O04Hgr1skO/9dNka4Ql2t4m4XbYEQX4ubzLIoA9KMysQQLw+?= =?us-ascii?Q?8ZYZEa7hUOXAfukIUHhGnVxz8gvf64r8Ao/nsWFnLu9MPdGqSc6riGwMZ7ow?= =?us-ascii?Q?p8fXrpXnus6SOCGu7z09vNvDbXUjuthh+NCe1hClbc9EPf1gG1SDrEL+6hVv?= =?us-ascii?Q?WpgGX4h7ylmOplJCHgG2JJ/e0Y2idmlYeTL5p2MMnVihkJhPrWdxSlXG/qXH?= =?us-ascii?Q?wPIshSqXK87+SBsBH+QObBMLOAOaNby10AROxdEJRJYxSdV85x+iP5kmwt0c?= =?us-ascii?Q?0WLW1JYTpXVckGqPdVP6mN5/mFWttbScUAviEtRb0jTvJOk9gOJluhsT642d?= =?us-ascii?Q?EhGni2Gm2i5cto64U8J+gOaefsC0DOQVg/3OxxtB7qBVpu67VK0x3bwcuhH9?= =?us-ascii?Q?EoPMqXKACOtY4mWlwkBrIpJ9xqX9G2dlZhqzr98rt9/t1r4ViGvknISOyk1L?= =?us-ascii?Q?l5V5Xc1izaoGrt20ty2hJG864RNDBNcUniWYwuab0YXSP7WVCEZZLQNKHqoT?= =?us-ascii?Q?4gqrbS+xWTzQBOrTwfDCBe/c4g/9R8Ymp662xZAz?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: efbf29e5-145e-4e2f-c0e9-08dc05e4bab6 X-MS-Exchange-CrossTenant-AuthSource: IA1PR12MB6332.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Dec 2023 07:31:52.4203 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IZoRotxdnc/r/fUcuJ5sdEojhZE5BHR6NKHMgG/YqBSn1/cHtPl+HSPa1KJ3rKX22tjFfbWHlzAamwTY94hOkg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4436 X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ci-bounces@dpdk.org Hello, Consider an option to describe DTS test with testpmd and SCAPY plain text commands. For example: Scenario: - Configure UDP packet in SCAPY and a flow in testpmd. - Send UDP packet and validate testpmd output triggered by that packet. ```yaml phase_0: name: CONFIGURATION tg: | udp_pkt = Ether()/IP()/UDP(sport=31)/Raw('== TEST ==') print('packet is ready') dut: | start set verbose 1 flow create 0 ingress pattern eth / ipv4 / udp src is 31 / end actions queue index 1 / end result: dut: 'Flow rule #0 created' tg: 'packet is ready' phase_1: name: SEND and VALIDATE tg: sendp(udp_pkt, iface=pf0) result: dut: '- RSS queue=0x1 -' ``` Test described as a sequence of phases. Phase definition: ~~~~~~~~~~~~~~~~ ``` : # unique phase ID name: # phase name : # application APP1 commands ... : # application APPy commands ... : # application APPx commands result: # optional phase results verification section : # APPx expected output ... : # APPy expected output ``` - Application commands in a phase executed sequentially, in order of application IDs: commands executed before commands. - Application results in a phase validated sequentially, in order of application IDs: result validated before APPy result. - Application result is a regular expression. Test application definition: ~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``` : # unique application ID agent: # mandatory application type identifier: {testpmd|scapy} cmd: # optional application command template ``` Example: ```yaml dut: agent: testpmd cmd: 'dpdk-testpmd -a pci0 -- -i --rxq=4 --txq=4' tg: agent: scapy ``` Test commands do not bound to a specific setup. Therefore, testpmd commad line and SCAPY sendp() function use encoding to describe relative interface position in a tested HBA. PCI encoding scheme for testpmd: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - PF PCI: `pciX` Example: `pci0: '0000:08:00.0'` - PCI SR-IOV: `pciXvfY` Example: `pci0vf0: '0000:08:00.2'` Network devices encoding scheme for SCAPY: - PF: `pfX` Example: `pf0: enp8s0f0np0` - PCI SR-IOV: `pfXvfY` Example: `pf0vf0: enp5s0f0v0` - Network device representor: `pfXrfY` Example: `pf0rf0: enp5s0f0npf0vf0` Test execution requires an additional file to describe tested setup. Setup file format: ~~~~~~~~~~~~~~~~~ ``` : # unique application ID host: # hostname or IPvX address path: # optional application path hba: # optional HBA description pmd: # PMD hw: # HW type ``` Example: ```yaml dut: host: 1.2.3.4 path: /opt/dpdk.org/build/app hba: pmd: mlx5 hw: mt4125 tg: host: ::1234 ``` ```yaml dut: agent: testpmd cmd: 'dpdk-testpmd -a pci0 -- -i --rxq=4 --txq=4' tg: agent: scapy test: - phases: [ *ref_phase0 ] repeat: 1 - phases: [ *ref_phase1 ] repeat: 3 phase_0: &ref_phase0 name: CONFIGURATION tg: | udp_pkt = Ether()/IP()/UDP(sport=31)/Raw('== TEST ==') print('packet is ready') dut: | start set verbose 1 flow create 0 ingress pattern eth / ipv4 / udp src is 31 / end actions queue index 1 / end result: dut: 'Flow rule #0 created' tg: 'packet is ready' phase_1: &ref_phase1 name: SEND and VALIDATE tg: sendp(udp_pkt, iface=pf0) result: dut: '- RSS queue=0x1 -' ``` The plain text format provides minimalistic and intuitive framework for DTS tests. DTS can use plan text testpmd/scapy command format in addition to Python framework. Regards, Gregory