Re: [dpdk-dev] [PATCH 05/13] app/test: introduce librte_security tests

[Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>]

Dear Akhil,

W dniu 01.04.2020 o 19:09, Akhil Goyal pisze:
> Hi Lukasz,
>
>
>> This patch introduces set of unit tests of librte_security API functions.
>> Tests are added to dpdk-test application and can be run with
>> "security_autotest" runtime command.
>>
>> This is the first patch in the series of patches as adding all test cases
>> for all API functions in a single patch would make it unreadable.
>>
>> This patch defines structure of the file and necessary test framework
>> initialization. It also contains first subset of unit tests for
>> rte_security_session_create API function.
>>
>> Structure of the tests file is following:
>> - macros for making tests more readable;
>> - mockup structures and functions for rte_security_ops;
>> - test suite and test cases setup and teardown functions;
>> - tests functions;
>> - declaration of testcases.
>>
>> Signed-off-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>
>> Change-Id: I3a4585f56ef1a75b2984fcea3a3e4a30e4c2d8a6
>> ---
> This patchset has a lot of repeated(for each API) tests just to check the input parameters to
> Rte_security APIs. I am not sure what value addition is done to separate out each API as a separate
> Negative Test. Instead a single case can be added to test all APIs with inappropriate arguments.
> We should add more positive cases with proper session parameters.
>
> Thomas,
> Do we allow these type of test cases in other modules?
>
> Regards,
> Akhil
As I already replied to you and Thomas, my primary intention was to make 
fixes to rte_security lib.

And as I did, I wanted to cover them also with unit tests. And as these 
are unit tests, there is plenty of them and all they do is to verify 
input parameters and check if proper functions are running and the code 
exists function in proper places with proper return values. So they 
verify the flow inside each API function. In my opinion it is best to 
have a single testcase to check single function flow with set 
conditions. So if anything fails in the future, you will know exactly 
what and why.

As the unit tests verify flow of the function, usualy they can fail in 
many ways, but the "proper" path to execute a function is only one, so 
there are much less positive testcases. Maybe more positive testcases 
you can see in the last patch containing tests for 
rte_security_capability_get. There are many testcases showing matches of 
capabilities to different patterns.

What you would like to have are functional tests not only for 
librte_security but together with its usage. I agree that would be nice 
to have such tests also, but those wouldn't test rte_security by itself, 
they would test its usage and that's completely different thing and 
that's not what these patches provide.

I'll check more modules to see how the tests look like there and I'll 
take a look at eventdev-tests, that Thomas mentioned.


Best regards

Lukasz

-- 

Lukasz Wojciechowski
Principal Software Engineer

Samsung R&D Institute Poland
Samsung Electronics
Office +48 22 377 88 25
l.wojciechow@partner.samsung.com