Re: [dpdk-dev] [PATCH v2 2/2] test/compress: im buffer too small - add unit tests

["Trahe, Fiona" <fiona.trahe@intel.com>]

> -----Original Message-----
> From: Akhil Goyal <akhil.goyal@nxp.com>
> Sent: Thursday, April 16, 2020 11:25 AM
> To: Trahe, Fiona <fiona.trahe@intel.com>; Dybkowski, AdamX <adamx.dybkowski@intel.com>;
> dev@dpdk.org
> Cc: Shally Verma <shallyv@marvell.com>
> Subject: RE: [PATCH v2 2/2] test/compress: im buffer too small - add unit tests
> 
> Hi Fiona,
> >
> > Hi Akhil,
> >
> > >
> > > Hi Fiona/Adam,
> > >
> > > > This patch adds a new test suite for verification of the "internal
> > > > QAT IM buffer too small" case handling. These unit tests are
> > > > specific to the QAT PMD only - that's why they are contained in
> > > > a separate test suite.
> > > >
> > > > Signed-off-by: Adam Dybkowski <adamx.dybkowski@intel.com>
> > > > ---
> > >
> > > Why do we need to have separate testsuite for QAT?
> > > Can't we have a single one and based on capability of the driver,
> > > Determine which tests need to be skipped in case they are not supported.
> > > This would create a mess in the longer run just like cryptodev.
> > >
> > > Please fix this, we cannot take this patch as is.
> >
> > [Fiona] Yes, I understand your concern and we considered including in the main
> > suite.
> > However these tests are not based on something that can be
> > checked in capabilities. They are tests to hone in on a specific corner case
> > based on a QAT limitation in its intermediate buffer size. So some of the
> > tests are to validate that the recent changes we made in the PMD correctly
> > work around that limitation, but other tests are negative and expected to fail
> > as provoking a corner-case that still exists. Other devices would probably not fail
> > the same tests.
> 
> Does that mean that all PMDs will pass with the newly added testcase which is for
> A corner case in QAT. If that is the case what is the issue in adding that in the main
> Test suite. It will get passed in all PMDs, isn't it? Am I missing something?
> 
> I believe we should not have PMD specific test suites, rather it should be based on
> Capabilities to identify the cases which should be run for that particular PMD.
[Fiona] yes, several of the cases should pass on all PMDs.
So we could move those into the main suite.
But what to do about the negative tests? 
Example: If a very large data buffer is passed to QAT to compress with dyn compression, it will get
split in the PMD into many smaller requests to the hardware. However if the number 
of requests is bigger than can fit on the qp then this will never succeed. The test
validates that the PMD behaves appropriately in this expected error case. That same
case would probably not have an error on another device. Maybe we should just leave out
such negative tests, but I find them useful as they validate the known behaviour.
The buffer size used in the test is based on the known size QAT can handle and the 
corner case in which QAT will return an error.

I see 4 options to handle this:
1. Leave out those tests
2. Use a qat-specific test suite only for negative cases which are constructed based on specific qat internal meta-data.
3. Include the negative tests in the main suite, but only run them on QAT (by checking driver type)
4. include the negative tests in the main suite, run them on all, expecting a FAIL from QAT and a PASS from other devices.

My preference is for 2.
But up to you.