Re: [PATCH v4 00/10] dts: add hello world testcase

[Bruce Richardson <bruce.richardson@intel.com>]

On Mon, Feb 13, 2023 at 04:28:36PM +0100, Juraj Linkeš wrote:
> Add code needed to run the HelloWorld testcase which just runs the hello
> world dpdk application.
> 
> The patchset currently heavily refactors this original DTS code needed
> to run the testcase:
> * The whole architecture has been redone into more sensible class
>   hierarchy
> * DPDK build on the System under Test
> * DPDK eal args construction, app running and shutting down
> * Optional SUT hugepage memory configuration
> * Test runner
> * Test results
> * TestSuite class
> * Test runner parts interfacing with TestSuite
> * The HelloWorld testsuite itself
> 
> The code is divided into sub-packages, some of which are divided
> further.
> 
> There patch may need to be divided into smaller chunks. If so, proposals
> on where exactly to split it would be very helpful.
> 
> v3:
> Finished refactoring everything in this patch, with test suite and test
> results being the last parts.
> Also changed the directory structure. It's now simplified and the
> imports look much better.
> I've also many many minor changes such as renaming variables here and
> there.
> 
> v4:
> Made hugepage config optional, users may now specify that in the main
> config file.
> Removed HelloWorld test plan and incorporated parts of it into the test
> suite python file.
> Updated documentation.
> 
Hi,

just trying this out by reading the docs and trying to follow along. Couple
of high-level comments thus far without getting into the patches:

* In the "configuring DTS" section, I think it would be good to:
   - say that the config file should be named conf.yaml by default. It's in
     the next section, but I think it should be called out earlier.
   - say that there is a template conf.yaml file in the dts directory
     already. On my first reading I actually thought that the sample config
     file was dts/framework/config/conf_yaml_schema.json (and I was going
     to comment on the name being weird! :-)). Only when I opened it did I
     realise my mistake. Therefore, downplan the schema, and put more
     emphasis on where to find the simple conf example to start with.
   - if hugepage config is now optional, as you say above, remove that from
     the sample and the docs.

* The code thus far seems to imply that you are always going to use root.
  When I configured it to log on to bruce@localhost, it timed out waiting
  for a prompt, I believe because it was looking for "#" which is the
  default only for root prompts.

* When running as root, things progressed further but I hit an error when
  DTS was trying to get the CPU config. No idea what is happening here,
  because running the same commands manually over ssh seemed to work fine.
  Below is the error. Any hints as to what is the problem appreciated.

Thanks,
/Bruce

$ ./main.py --tarball ~/Downloads/dpdk-22.11.1.tar.xz -v Y
2023/02/17 16:59:57 - SUT 1 - INFO - Connecting to root@localhost.
2023/02/17 16:59:58 - SUT 1 - INFO - Connection to root@localhost successful.
2023/02/17 16:59:58 - SUT 1 - INFO - Getting CPU information.
2023/02/17 16:59:58 - SUT 1 - INFO - Sending: 'lscpu -p=CPU,CORE,SOCKET,NODE|grep -v \#'
2023/02/17 16:59:59 - dts_runner - ERROR - Connection to node NodeConfiguration(name='SUT 1', hostname='localhost', user='root', password=None, arch=<Architecture.x86_64: 'x86_64'>, os=<OS.linux: 'linux'>, lcores='3,4', use_first_core=False, memory_channels=8, hugepages=HugepageConfiguration(amount=256, force_first_numa=False)) failed.
Traceback (most recent call last):
  File "/home/bruce/dpdk.org/dts/framework/dts.py", line 41, in run_all
    sut_node = SutNode(execution.system_under_test)
  File "/home/bruce/dpdk.org/dts/framework/testbed_model/sut_node.py", line 39, in __init__
    super(SutNode, self).__init__(node_config)
  File "/home/bruce/dpdk.org/dts/framework/testbed_model/node.py", line 47, in __init__
    self._get_remote_cpus()
  File "/home/bruce/dpdk.org/dts/framework/testbed_model/node.py", line 155, in _get_remote_cpus
    self.lcores = self.main_session.get_remote_cpus(self.config.use_first_core)
  File "/home/bruce/dpdk.org/dts/framework/remote_session/linux_session.py", line 18, in get_remote_cpus
    cpu_info = self.remote_session.send_command(
  File "/home/bruce/dpdk.org/dts/framework/remote_session/remote/remote_session.py", line 103, in send_command
    result = self._send_command(command, timeout, env)
  File "/home/bruce/dpdk.org/dts/framework/remote_session/remote/ssh_session.py", line 172, in _send_command
    return_code = int(self._send_command_get_output("echo $?", timeout, None))
ValueError: invalid literal for int() with base 10: '\x1b[?2004l\r\r\n0'
2023/02/17 16:59:59 - dts_runner - DEBUG - Summary of errors:
2023/02/17 16:59:59 - dts_runner - DEBUG - ValueError("invalid literal for int() with base 10: '\\x1b[?2004l\\r\\r\\n0'")
2023/02/17 16:59:59 - dts_runner - INFO - DTS execution has ended.