Re: [dpdk-dev] [PATCH v3 3/5] bbdev: test applications

["Mokhtar, Amr" <amr.mokhtar@intel.com>]

> -----Original Message-----
> From: Yigit, Ferruh
> Sent: Monday 11 December 2017 19:01
> To: Mokhtar, Amr <amr.mokhtar@intel.com>; dev@dpdk.org
> Cc: thomas@monjalon.net; Burakov, Anatoly <anatoly.burakov@intel.com>; De
> Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>; Power, Niall
> <niall.power@intel.com>; Macnamara, Chris <chris.macnamara@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v3 3/5] bbdev: test applications
> 
> On 12/7/2017 1:40 PM, Amr Mokhtar wrote:
> > - Full test suite for bbdev
> > - Test App works seamlessly on all PMDs registered with bbdev
> >  framework
> > - A python script is provided to make our life easier
> 
> Can you please describe what the script is for?

In order to run a test many parameters are required, like EAL, test vectors to use
the type of test (latency, offload, throughput), the binary application location,
number of operations, bursts size, etc. The script eases all that, and assumes
default values, in case they were not provided by the tester.
In its simplest form, it can execute as a smoke test by executing:
$ ./test-bbdev.py

This will run a default test with default parameters on bbdev_null device.

> 
> > - Supports execution of tests by parsing Test Vector files
> 
> Can you please describe what are test vector files?

Test vectors are set of input and output parameters when combined together
form one test scenario. These values are logically correlated to provide
a realistic test scenario. Furthermore, they include the data in the format that 
matches the module under test.

In our case, we are concerned of testing various Turbo coding scenarios, so when
testing a decode operation for example, the input data in the test should match
the Turbo encoded data as specified in 3GPP TS 36.212 specs, namely the "virtual
circular buffer."

Test-bbdev application parses the parameters from the test vector file into its
local memory to form the op (operation) structure and loads the data into the
input mbufs and saves the expected output. It enqueues the op structures, 
dequeues them, then compares the computed output with the expected output.
If matching, the test is passed.

> <...>
> 
> > +include $(RTE_SDK)/mk/rte.vars.mk
> > +
> > +ifeq ($(CONFIG_RTE_TEST_BBDEV),y)
> 
> You don't need this ifdef I think, although I can see testpmd has it ...

Corrected.

> 
> > +
> > +#
> > +# library name
> > +#
> > +APP = testbbdev
> > +
> > +CFLAGS += -O3
> > +CFLAGS += $(WERROR_FLAGS)
> > +
> > +#
> > +# all sources are stored in SRCS-y
> > +#
> > +SRCS-$(CONFIG_RTE_LIBRTE_BBDEV) += main.c
> > +SRCS-$(CONFIG_RTE_LIBRTE_BBDEV) += test_bbdev.c
> > +SRCS-$(CONFIG_RTE_LIBRTE_BBDEV) += test_bbdev_perf.c
> > +SRCS-$(CONFIG_RTE_LIBRTE_BBDEV) += test_bbdev_vector.c
> 
> If you remove above wrapping ifdef, you may use CONFIG_RTE_TEST_BBDEV
> instead.

Corrected.
 
> <...>
> 
> > +int
> > +unit_test_suite_runner(struct unit_test_suite *suite)
> 
> Is test-bbdev application a suit of test cases? What is the benefit of having
> separate application comparing adding unit tests to test/ folder?

Yes it is a suite of test ceses.
The benefit is to have all test applications self-contained and separate.
So, every device abstraction library has its own test-xxx folder.

> <...>