RE: DTS testpmd and SCAPY integration

[Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>]

> -----Original Message-----
> From: Etelson, Gregory <getelson@nvidia.com>
> Sent: Tuesday, December 26, 2023 1:32 AM
> To: thomas@monjalon.net; Juraj Linkeš <juraj.linkes@pantheon.tech>;
> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; Paul Szczepanek
> <Paul.Szczepanek@arm.com>; Luca Vizzarro <Luca.Vizzarro@arm.com>; Yoan
> Picchi <yoan.picchi@foss.arm.com>; Jeremy Spewock
> <jspewock@iol.unh.edu>; Gregory Etelson <getelson@nvidia.com>; Patrick
> Robb <probb@iol.unh.edu>; ci@dpdk.org; dev@dpdk.org
> Subject: DTS testpmd and SCAPY integration
> 
> Hello,
> 
> Consider an option to describe DTS test with testpmd and SCAPY plain text
> commands.
> 
> For example:
> 
> Scenario:
>   - Configure UDP packet in SCAPY and a flow in testpmd.
>   - Send UDP packet and validate testpmd output triggered by that packet.
> 
> ```yaml
> phase_0:
>      name: CONFIGURATION
>      tg: |
>          udp_pkt = Ether()/IP()/UDP(sport=31)/Raw('== TEST ==')
>          print('packet is ready')
>      dut: |
>          start
>          set verbose 1
>          flow create 0 ingress pattern eth / ipv4 / udp src is 31 / end actions queue
> index 1 / end
>      result:
>          dut: 'Flow rule #0 created'
>          tg: 'packet is ready'
> 
> phase_1:
>      name: SEND and VALIDATE
>      tg: sendp(udp_pkt, iface=pf0)
>      result:
>          dut: '- RSS queue=0x1 -'
> ```
> 
> Test described as a sequence of phases.
> 
> Phase definition:
> ~~~~~~~~~~~~~~~~
> 
> ```
> <phase id>: # unique phase ID
>      name: <phase name>                       # phase name
>      <application ID (APP1)>: <APP1 commands> # application APP1 commands
>      ...
>      <application ID (APPy)>: <APPy commands> # application APPy commands
>      ...
>      <application ID (APPx)>: <APPx commands> # application APPx commands
> 
>      result:                # optional phase results verification section
>          <application ID APPx>: <expected output>  # APPx expected output
>          ...
>          <application ID APPy>: <expected output>  # APPy expected output ```
> 
> - Application commands in a phase executed sequentially,
>    in order of application IDs: <APPy> commands executed
>    before <APPx> commands.
> 
> - Application results in a phase validated sequentially,
>    in order of application IDs: <APPx> result validated
>    before APPy result.
> 
> - Application result is a regular expression.
> 
> 
> Test application definition:
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> ```
> <application ID>: # unique application ID
>      agent:        # mandatory application type identifier: {testpmd|scapy}
>      cmd:          # optional application command template
> ```
> 
> Example:
> 
> ```yaml
> 
> dut:
>      agent: testpmd
>      cmd: 'dpdk-testpmd -a pci0 -- -i --rxq=4 --txq=4'
> 
> tg:
>      agent: scapy
> ```
> 
> Test commands do not bound to a specific setup.
> Therefore, testpmd commad line and SCAPY sendp() function use encoding to
> describe relative interface position in a tested HBA.
> 
> PCI encoding scheme for testpmd:
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> - PF PCI: `pciX`
>    Example: `pci0: '0000:08:00.0'`
> 
> 
> - PCI SR-IOV: `pciXvfY`
>    Example: `pci0vf0: '0000:08:00.2'`
> 
> Network devices encoding scheme for SCAPY:
> - PF: `pfX`
>    Example: `pf0: enp8s0f0np0`
> 
> 
> - PCI SR-IOV: `pfXvfY`
>    Example: `pf0vf0: enp5s0f0v0`
> 
> 
> - Network device representor: `pfXrfY`
> 
>       Example: `pf0rf0: enp5s0f0npf0vf0`
> 
> 
> Test execution requires an additional file to describe tested setup.
> 
> Setup file format:
> ~~~~~~~~~~~~~~~~~
> 
> ```
> <application id>: # unique application ID
>      host:         # hostname or IPvX address
>      path:         # optional application path
>      hba:          # optional HBA description
>          pmd:      # PMD
>          hw:       # HW type
> ```
> 
> Example:
> 
> ```yaml
> dut:
>      host: 1.2.3.4
>      path: /opt/dpdk.org/build/app
>      hba:
>          pmd: mlx5
>          hw: mt4125
> tg:
>      host: ::1234
> ```
> 
> ```yaml
> dut:
>      agent: testpmd
>      cmd: 'dpdk-testpmd -a pci0 -- -i --rxq=4 --txq=4'
> tg:
>      agent: scapy
> 
> test:
>      -
>          phases: [ *ref_phase0 ]
>          repeat: 1
>      -
>          phases: [ *ref_phase1 ]
>          repeat: 3
> 
> phase_0: &ref_phase0
>      name: CONFIGURATION
>      tg: |
>          udp_pkt = Ether()/IP()/UDP(sport=31)/Raw('== TEST ==')
>          print('packet is ready')
>      dut: |
>          start
>          set verbose 1
>          flow create 0 ingress pattern eth / ipv4 / udp src is 31 / end actions queue
> index 1 / end
>      result:
>          dut: 'Flow rule #0 created'
>          tg: 'packet is ready'
> 
> phase_1: &ref_phase1
>      name: SEND and VALIDATE
>      tg: sendp(udp_pkt, iface=pf0)
>      result:
>          dut: '- RSS queue=0x1 -'
> ```
> 
> The plain text format provides minimalistic and intuitive framework for DTS
> tests.
> DTS can use plan text testpmd/scapy command format in addition to Python
> framework.
Hi Gregory,
	I do not fully understand your proposal, it will be helpful to join the DTS meetings to discuss this further.

YAML has wide support built around it. By using our own text format, we will have to build the parsing support etc ourselves.

However, YAML is supposed to be easy to read and understand. Is it just a matter for getting used to it?

Thank you,
Honnappa

> 
> Regards,
> Gregory