Re: [dpdk-dev] [PATCH] app/test: enhance sanity script

["Burakov, Anatoly" <anatoly.burakov@intel.com>]

On 27-Apr-18 2:20 PM, Reshma Pattan wrote:
> 1. Sanity Script is enhanced with additional test cases
> being added to autotest_data.py
> 2. Fixed in autotest_test_funcs.py to handle test cases
> which returns "Skipped" as result.
>    The issue was skipped test cases got timed out,
> causing delay in sanity script execution.
> 3. Enhanced support for FreeBSD to add the respective memory limit,
> as FreeBSD doesn't support socket-mem, file-prefix options.
> 
> Signed-off-by: Jananee Parthasarathy <jananeex.m.parthasarathy@intel.com>
> ---

I don't think number 3 on that list is needed. FreeBSD doesn't support 
file-prefixes, so it's not possible to run multiple unit tests in 
parallel in the first place, so no need for grouping and no need to 
limit allocated memory.

On a more general note, IMO this whole thing needs to be redone and 
simplified. I wanted to fix all this as part of Memory Hotplug patches, 
but didn't get to it yet.

(not to mention that this infrastructure itself is a lot like IVSHMEM, 
in that it was built to solve a specific problem, consequences be damned...)

The reasoning behind splitting tests into groups was largely the fact 
that we had to preallocate memory, while also running tests in parallel. 
This is no longer needed for two reasons.

First, most (if not all) tests now clean up properly after themselves 
(when this was written, it wasn't possible to un-reserve memzones, rings 
etc. - now it is), so we can run them one by one without restarting the 
test application.

Second, memory is now allocated on the fly, so there is no more reason 
to split autotests into groups at all. Instead, we can just group all 
tests into "performance" (to be run one by one) and "non performance" 
(can be run concurrently via different prefixes), and run them with a 
pool of workers.

For FreeBSD, since there's no dynamic memory allocation and no support 
for file prefixes, there needs to be only one worker, running all tests 
one by one. For Linux, you can add all unit tests into a queue, and use 
a pool of workers dequeuing and doing tests in parallel under different 
prefixes. No '-m' or '--socket-mem' switches needed.

-- 
Thanks,
Anatoly