[PATCH] test: raise fast test timeout to 60s on RISC-V

[PATCH] test: raise fast test timeout to 60s on RISC-V

[Eric Long]

Current RISC-V hardware (e.g. HiFive Unmatched) is still way too slow
compared to other architectures' counterparts. On the most powerful
RISC-V CPU available (SG2042), DPDK still fails with 4 timeouts:

```
Summary of Failures:

23/77 DPDK:fast-tests / eventdev_selftest_sw        TIMEOUT
    10.06s   killed by signal 15 SIGTERM
56/77 DPDK:fast-tests / rib6_autotest               TIMEOUT
    10.04s
64/77 DPDK:fast-tests / spinlock_autotest           TIMEOUT
    10.06s   killed by signal 15 SIGTERM
66/77 DPDK:fast-tests / table_autotest              TIMEOUT
    11.28s   killed by signal 15 SIGTERM
```

On HiFive Unmatched, the longest test takes 53 seconds:

```
37/77 DPDK:fast-tests / lpm6_autotest               OK
    53.29s
```

Raising timeout to 60s on RISC-V target will help test effectiveness
on it.

Signed-off-by: Eric Long <i@hack3r.moe>
---
 .mailmap                    | 1 +
 app/test/suites/meson.build | 5 +++++
 2 files changed, 6 insertions(+)

diff --git a/.mailmap b/.mailmap
index 7334ad58a9..24f3f4a0cd 100644
--- a/.mailmap
+++ b/.mailmap
@@ -404,6 +404,7 @@ Erez Ferber <erezf@nvidia.com> <erezf@mellanox.com>
 Erez Shitrit <erezsh@nvidia.com>
 Eric Joyner <eric.joyner@intel.com>
 Eric Kinzie <ekinzie@brocade.com> <ehkinzie@gmail.com>
+Eric Long <i@hack3r.moe>
 Eric Zhang <eric.zhang@windriver.com>
 Erik Gabriel Carrillo <erik.g.carrillo@intel.com>
 Erik Ziegenbalg <eziegenb@brocade.com>
diff --git a/app/test/suites/meson.build b/app/test/suites/meson.build
index 191702cf76..b74d1c4ad7 100644
--- a/app/test/suites/meson.build
+++ b/app/test/suites/meson.build
@@ -6,6 +6,11 @@
 timeout_seconds = 600
 timeout_seconds_fast = 10
 
+if arch_subdir == 'riscv'
+    # Current RISC-V machines are too slow to finish fast tests under 10s
+    timeout_seconds_fast = 60
+endif
+
 test_no_huge_args = ['--no-huge', '-m', '2048']
 has_hugepage = run_command(has_hugepages_cmd, check: true).stdout().strip() != '0'
 message('hugepage availability: @0@'.format(has_hugepage))
-- 
2.47.0

Re: [PATCH] test: raise fast test timeout to 60s on RISC-V

[David Marchand]

On Sat, Nov 23, 2024 at 4:00 PM Eric Long <i@hack3r.moe> wrote:
>
> Current RISC-V hardware (e.g. HiFive Unmatched) is still way too slow
> compared to other architectures' counterparts. On the most powerful
> RISC-V CPU available (SG2042), DPDK still fails with 4 timeouts:
>
> ```
> Summary of Failures:
>
> 23/77 DPDK:fast-tests / eventdev_selftest_sw        TIMEOUT
>     10.06s   killed by signal 15 SIGTERM
> 56/77 DPDK:fast-tests / rib6_autotest               TIMEOUT
>     10.04s
> 64/77 DPDK:fast-tests / spinlock_autotest           TIMEOUT
>     10.06s   killed by signal 15 SIGTERM
> 66/77 DPDK:fast-tests / table_autotest              TIMEOUT
>     11.28s   killed by signal 15 SIGTERM
> ```
>
> On HiFive Unmatched, the longest test takes 53 seconds:
>
> ```
> 37/77 DPDK:fast-tests / lpm6_autotest               OK
>     53.29s
> ```
>
> Raising timeout to 60s on RISC-V target will help test effectiveness
> on it.

You can extend the timeout via the multiplier option (default timeout
of 10s * multiplier).
So in your case:
$ meson test -C <build> --suite fast-tests -t 6


-- 
David Marchand

Re: [PATCH] test: raise fast test timeout to 60s on RISC-V

[Eric Long]

On 27/11/2024 04:29, David Marchand wrote:
> You can extend the timeout via the multiplier option (default timeout
> of 10s * multiplier).
> So in your case:
> $ meson test -C <build> --suite fast-tests -t 6

I hope the RISC-V specific extended timeout could be upstreamed though, 
in this way we won't need to bump timeout in every distro supporting the 
architecture (like Debian [1]) or even not running the tests altogether 
(like OpenSUSE [2] and Fedora [3]), just because tests are failing due 
to timeout.

Cheers,
Eric

[1]: 
https://salsa.debian.org/debian/dpdk/-/blob/unstable/debian/tests/test-fastsuite?ref_type=heads#L31
[2]: 
https://build.opensuse.org/projects/openSUSE:Factory/packages/dpdk/files/dpdk.spec?expand=1
[3]: https://src.fedoraproject.org/rpms/dpdk/blob/rawhide/f/dpdk.spec