[dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework

DPDK patches and discussions
 help / color / mirror / Atom feed

* [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework
@ 2021-04-23 16:18 Ciara Power
  2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 1/7] app/test: refactor of unit test suite runner Ciara Power
                   ` (9 more replies)
  0 siblings, 10 replies; 15+ messages in thread
From: Ciara Power @ 2021-04-23 16:18 UTC (permalink / raw)
  To: dev
  Cc: thomas, declan.doherty, gakhil, aconole, hemant.agrawal, anoobj,
	ruifeng.wang, asomalap, ajit.khaparde, g.singh, Ciara Power

The current crypto unit test framework is not granular enough to
accurately track unit test results. This is caused by one testcase
in a suite actually running multiple testcases, but only returning
one result.

The approach taken in this patchset allows a test suite have a
list of sub-testsuites, and/or a list of testcases as previously used.
The unit test suite runner can then recursively iterate and run the
sub-testsuites, until it reaches a suite with testcases,
and it then runs each testcase as it had done previously.
In the case of a testsuite with both testcases and sub-testsuites,
the testcases are executed first before iterating through the
sub-testsuites.

By allowing this further breakdown into sub-testsuites,
a refactor of the crypto unit tests solves the issue of inaccurate
reporting, as sub-testsuites can be used in place of the testcases
that had multiple testcases hidden on a sub level.
The blockcipher tests previously had these hidden testcases,
but are now sub-testsuites that are dynamically created and added to a
parent test suite, allowing for each testcase status to be reported
directly to the runner.
The cryptodev test suite is broken down into smaller suites that are
used as sub-testsuites, which allows for more flexibility choosing which
sub-testsuites should run for the current device.
The introduction of sub-testsuites also allows for more precise
setup/teardown functions, that can check the capabilities required to
run its testcases.

For example, when running the cryptodev_aesni_mb_autotest, 
the parent Cryptodev Test Suite is executed.
Various sub-testsuites are added to the parent test suite, such as
the static suites of testcases that were once in the cryptodev_testsuite,
and blockcipher suites.
The unit test runner can then run the Cryptodev parent test suite,
which in turn will run the sub-testsuites.

The user is now required to create vdevs via EAL commandline args,
this is no longer done within the test app for crypto autotests.

Documentation will need to be added at a later stage,
adding to the test document that isn't yet merged. [1]

[1] https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536-1-aconole@redhat.com/

v3:
  - Added support for a testsuite having both a list of testcases,
    and a list of sub-testsuites.
  - Replaced PMD based parent testsuites with a cryptodev testsuite
    used by all autotests, with the exception of scheduler autotest.
  - Setup functions were added for all sub-testsuites, within which
    required capability support is checked.
  - The setup functions no longer create vdevs if needed,
    this must be done by the user when running the test.
  - Patch added to standardise return values for skipped testcases.
v2:
  - Added macro in place of testcase/testsuite loops.
  - Added more detail in the summary output.
  - Moved testcase counts to the testsuite structure.
  - Flattened testsuite structure to remove union.
  - Added patch for fix of blockcipher test return value.
  - Squashed release note into last patch.

Ciara Power (7):
  app/test: refactor of unit test suite runner
  test: introduce parent testsuite format
  test/crypto: refactor to use sub-testsuites
  test/crypto: replace unsupported with skipped
  test/crypto: move testsuite params to header file
  test/crypto: fix return value on test skipped
  test/crypto: dynamically build blockcipher suite

 app/test/test.c                        |  176 +-
 app/test/test.h                        |   23 +-
 app/test/test_cryptodev.c              | 2324 ++++++++++++++----------
 app/test/test_cryptodev.h              |   32 +
 app/test/test_cryptodev_asym.c         |  111 +-
 app/test/test_cryptodev_blockcipher.c  |  423 ++++-
 app/test/test_cryptodev_blockcipher.h  |   12 +-
 app/test/test_ipsec.c                  |   32 +-
 doc/guides/rel_notes/release_21_05.rst |    5 +
 9 files changed, 2000 insertions(+), 1138 deletions(-)

-- 
2.25.1

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [dpdk-dev] [PATCH v3 1/7] app/test: refactor of unit test suite runner
  2021-04-23 16:18 [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework Ciara Power
@ 2021-04-23 16:18 ` Ciara Power
  2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 2/7] test: introduce parent testsuite format Ciara Power
                   ` (8 subsequent siblings)
  9 siblings, 0 replies; 15+ messages in thread
From: Ciara Power @ 2021-04-23 16:18 UTC (permalink / raw)
  To: dev
  Cc: thomas, declan.doherty, gakhil, aconole, hemant.agrawal, anoobj,
	ruifeng.wang, asomalap, ajit.khaparde, g.singh, Ciara Power

Some small changes were made to the unit test suite runner for
readability and to enable reuse of some of the function in a later patch.

On test suite setup skip/fail, the loop to count testcases as
skipped/failed has been moved to another function.
This will allow for recursion in a later patch when nested sub-testsuites
are used.

The unit test suite runner accessed the list of testcases in the suite
structure every time the testcase was used. This is now replaced by a
testcase variable which improves readability.

A macro has been introduced for readability, instead of using open
coded loops.

Rather than keep local variable status counts for testcases,
these are added to the test suite structure.

The summary output now prints the suite name, this will be useful later
when multiple nested sub-testsuites are being run.

Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: Aaron Conole <aconole@redhat.com>

---
v3:
  - Fixed index used when counting testcases on setup fail.
v2:
  - Added macro to loop testcases in suite.
  - Testcase counts added to the test suite structure.
---
 app/test/test.c | 103 +++++++++++++++++++++++++++++-------------------
 app/test/test.h |   6 +++
 2 files changed, 69 insertions(+), 40 deletions(-)

diff --git a/app/test/test.c b/app/test/test.c
index 94352a95d5..2fb99d0855 100644
--- a/app/test/test.c
+++ b/app/test/test.c
@@ -36,6 +36,11 @@ extern cmdline_parse_ctx_t main_ctx[];
 
 #define RTE_LOGTYPE_APP RTE_LOGTYPE_USER1
 
+#define FOR_EACH_SUITE_TESTCASE(iter, suite, case)			\
+	for (iter = 0, case = suite->unit_test_cases[0];		\
+		suite->unit_test_cases[iter].testcase;			\
+		iter++, case = suite->unit_test_cases[iter])
+
 const char *prgname; /* to be set to argv[0] */
 
 static const char *recursive_call; /* used in linux for MP and other tests */
@@ -234,14 +239,41 @@ main(int argc, char **argv)
 	return ret;
 }
 
+static void
+unit_test_suite_count_tcs_on_setup_fail(struct unit_test_suite *suite,
+		int test_success)
+{
+	struct unit_test_case tc;
+	int i;
+
+	FOR_EACH_SUITE_TESTCASE(i, suite, tc) {
+		suite->total++;
+		if (!tc.enabled || test_success == TEST_SKIPPED)
+			suite->skipped++;
+		else
+			suite->failed++;
+	}
+}
+
+static void
+unit_test_suite_reset_counts(struct unit_test_suite *suite)
+{
+	suite->total = 0;
+	suite->executed = 0;
+	suite->succeeded = 0;
+	suite->skipped = 0;
+	suite->failed = 0;
+	suite->unsupported = 0;
+}
 
 int
 unit_test_suite_runner(struct unit_test_suite *suite)
 {
 	int test_success;
-	unsigned int total = 0, executed = 0, skipped = 0;
-	unsigned int succeeded = 0, failed = 0, unsupported = 0;
 	const char *status;
+	struct unit_test_case tc;
+
+	unit_test_suite_reset_counts(suite);
 
 	if (suite->suite_name) {
 		printf(" + ------------------------------------------------------- +\n");
@@ -255,55 +287,48 @@ unit_test_suite_runner(struct unit_test_suite *suite)
 			 * setup did not pass, so count all enabled tests and
 			 * mark them as failed/skipped
 			 */
-			while (suite->unit_test_cases[total].testcase) {
-				if (!suite->unit_test_cases[total].enabled ||
-				    test_success == TEST_SKIPPED)
-					skipped++;
-				else
-					failed++;
-				total++;
-			}
+			unit_test_suite_count_tcs_on_setup_fail(suite,
+					test_success);
 			goto suite_summary;
 		}
 	}
 
 	printf(" + ------------------------------------------------------- +\n");
 
-	while (suite->unit_test_cases[total].testcase) {
-		if (!suite->unit_test_cases[total].enabled) {
-			skipped++;
-			total++;
+	FOR_EACH_SUITE_TESTCASE(suite->total, suite, tc) {
+		if (!tc.enabled) {
+			suite->skipped++;
 			continue;
 		} else {
-			executed++;
+			suite->executed++;
 		}
 
 		/* run test case setup */
-		if (suite->unit_test_cases[total].setup)
-			test_success = suite->unit_test_cases[total].setup();
+		if (tc.setup)
+			test_success = tc.setup();
 		else
 			test_success = TEST_SUCCESS;
 
 		if (test_success == TEST_SUCCESS) {
 			/* run the test case */
-			test_success = suite->unit_test_cases[total].testcase();
+			test_success = tc.testcase();
 			if (test_success == TEST_SUCCESS)
-				succeeded++;
+				suite->succeeded++;
 			else if (test_success == TEST_SKIPPED)
-				skipped++;
+				suite->skipped++;
 			else if (test_success == -ENOTSUP)
-				unsupported++;
+				suite->unsupported++;
 			else
-				failed++;
+				suite->failed++;
 		} else if (test_success == -ENOTSUP) {
-			unsupported++;
+			suite->unsupported++;
 		} else {
-			failed++;
+			suite->failed++;
 		}
 
 		/* run the test case teardown */
-		if (suite->unit_test_cases[total].teardown)
-			suite->unit_test_cases[total].teardown();
+		if (tc.teardown)
+			tc.teardown();
 
 		if (test_success == TEST_SUCCESS)
 			status = "succeeded";
@@ -314,10 +339,8 @@ unit_test_suite_runner(struct unit_test_suite *suite)
 		else
 			status = "failed";
 
-		printf(" + TestCase [%2d] : %s %s\n", total,
-				suite->unit_test_cases[total].name, status);
-
-		total++;
+		printf(" + TestCase [%2d] : %s %s\n", suite->total,
+				tc.name, status);
 	}
 
 	/* Run test suite teardown */
@@ -328,20 +351,20 @@ unit_test_suite_runner(struct unit_test_suite *suite)
 
 suite_summary:
 	printf(" + ------------------------------------------------------- +\n");
-	printf(" + Test Suite Summary \n");
-	printf(" + Tests Total :       %2d\n", total);
-	printf(" + Tests Skipped :     %2d\n", skipped);
-	printf(" + Tests Executed :    %2d\n", executed);
-	printf(" + Tests Unsupported:  %2d\n", unsupported);
-	printf(" + Tests Passed :      %2d\n", succeeded);
-	printf(" + Tests Failed :      %2d\n", failed);
+	printf(" + Test Suite Summary : %s\n", suite->suite_name);
+	printf(" + Tests Total :       %2d\n", suite->total);
+	printf(" + Tests Skipped :     %2d\n", suite->skipped);
+	printf(" + Tests Executed :    %2d\n", suite->executed);
+	printf(" + Tests Unsupported:  %2d\n", suite->unsupported);
+	printf(" + Tests Passed :      %2d\n", suite->succeeded);
+	printf(" + Tests Failed :      %2d\n", suite->failed);
 	printf(" + ------------------------------------------------------- +\n");
 
-	last_test_result = failed;
+	last_test_result = suite->failed;
 
-	if (failed)
+	if (suite->failed)
 		return TEST_FAILED;
-	if (total == skipped)
+	if (suite->total == suite->skipped)
 		return TEST_SKIPPED;
 	return TEST_SUCCESS;
 }
diff --git a/app/test/test.h b/app/test/test.h
index cd047eb26c..55f2850fed 100644
--- a/app/test/test.h
+++ b/app/test/test.h
@@ -138,6 +138,12 @@ struct unit_test_suite {
 	const char *suite_name;
 	int (*setup)(void);
 	void (*teardown)(void);
+	unsigned int total;
+	unsigned int executed;
+	unsigned int succeeded;
+	unsigned int skipped;
+	unsigned int failed;
+	unsigned int unsupported;
 	struct unit_test_case unit_test_cases[];
 };
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* [dpdk-dev] [PATCH v3 2/7] test: introduce parent testsuite format
  2021-04-23 16:18 [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework Ciara Power
  2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 1/7] app/test: refactor of unit test suite runner Ciara Power
@ 2021-04-23 16:18 ` Ciara Power
  2021-04-23 16:32   ` Aaron Conole
  2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 3/7] test/crypto: refactor to use sub-testsuites Ciara Power
                   ` (7 subsequent siblings)
  9 siblings, 1 reply; 15+ messages in thread
From: Ciara Power @ 2021-04-23 16:18 UTC (permalink / raw)
  To: dev
  Cc: thomas, declan.doherty, gakhil, aconole, hemant.agrawal, anoobj,
	ruifeng.wang, asomalap, ajit.khaparde, g.singh, Ciara Power

The current structure for unit testing only allows for running a
test suite with nested test cases. This means all test cases for an
autotest must be in one suite, which is not ideal.
For example, in some cases we may want to run multiple lists of test
cases that each require different setup, so should be in separate suites.

The unit test suite struct is modified to hold a pointer to a list of
sub-testsuite pointers, along with the list of testcases as before.

Signed-off-by: Ciara Power <ciara.power@intel.com>

---
v3:
  - Added condition check to macro that loops sub-testsuites.
  - Changed execution to allow for mixture of sub-testsuites and
    testcases, by removing the if/else condition.
  - Moved testcase execution above sub-testsuite execution,
    to improve output readability.
v2:
  - Added macro to loop sub-testsuites.
  - Added sub-testsuite summary detail.
---
 app/test/test.c | 70 ++++++++++++++++++++++++++++++++++++++++++++++---
 app/test/test.h |  1 +
 2 files changed, 68 insertions(+), 3 deletions(-)

diff --git a/app/test/test.c b/app/test/test.c
index 2fb99d0855..ac0a66392a 100644
--- a/app/test/test.c
+++ b/app/test/test.c
@@ -41,6 +41,12 @@ extern cmdline_parse_ctx_t main_ctx[];
 		suite->unit_test_cases[iter].testcase;			\
 		iter++, case = suite->unit_test_cases[iter])
 
+#define FOR_EACH_SUITE_TESTSUITE(iter, suite, sub_ts)			\
+	for (iter = 0, sub_ts = suite->unit_test_suites ?		\
+		suite->unit_test_suites[0]:NULL; sub_ts &&		\
+		suite->unit_test_suites[iter]->suite_name != NULL;	\
+		iter++, sub_ts = suite->unit_test_suites[iter])
+
 const char *prgname; /* to be set to argv[0] */
 
 static const char *recursive_call; /* used in linux for MP and other tests */
@@ -241,11 +247,26 @@ main(int argc, char **argv)
 
 static void
 unit_test_suite_count_tcs_on_setup_fail(struct unit_test_suite *suite,
-		int test_success)
+		int test_success, unsigned int *sub_ts_failed,
+		unsigned int *sub_ts_skipped, unsigned int *sub_ts_total)
 {
 	struct unit_test_case tc;
+	struct unit_test_suite *ts;
 	int i;
 
+	FOR_EACH_SUITE_TESTSUITE(i, suite, ts) {
+		unit_test_suite_count_tcs_on_setup_fail(
+			ts, test_success, sub_ts_failed,
+			sub_ts_skipped, sub_ts_total);
+		suite->total += ts->total;
+		suite->failed += ts->failed;
+		suite->skipped += ts->skipped;
+		if (ts->failed)
+			(*sub_ts_failed)++;
+		else
+			(*sub_ts_skipped)++;
+		(*sub_ts_total)++;
+	}
 	FOR_EACH_SUITE_TESTCASE(i, suite, tc) {
 		suite->total++;
 		if (!tc.enabled || test_success == TEST_SKIPPED)
@@ -258,6 +279,11 @@ unit_test_suite_count_tcs_on_setup_fail(struct unit_test_suite *suite,
 static void
 unit_test_suite_reset_counts(struct unit_test_suite *suite)
 {
+	struct unit_test_suite *ts;
+	int i;
+
+	FOR_EACH_SUITE_TESTSUITE(i, suite, ts)
+		unit_test_suite_reset_counts(ts);
 	suite->total = 0;
 	suite->executed = 0;
 	suite->succeeded = 0;
@@ -269,9 +295,12 @@ unit_test_suite_reset_counts(struct unit_test_suite *suite)
 int
 unit_test_suite_runner(struct unit_test_suite *suite)
 {
-	int test_success;
+	int test_success, i, ret;
 	const char *status;
 	struct unit_test_case tc;
+	struct unit_test_suite *ts;
+	unsigned int sub_ts_succeeded = 0, sub_ts_failed = 0;
+	unsigned int sub_ts_skipped = 0, sub_ts_total = 0;
 
 	unit_test_suite_reset_counts(suite);
 
@@ -288,7 +317,8 @@ unit_test_suite_runner(struct unit_test_suite *suite)
 			 * mark them as failed/skipped
 			 */
 			unit_test_suite_count_tcs_on_setup_fail(suite,
-					test_success);
+					test_success, &sub_ts_failed,
+					&sub_ts_skipped, &sub_ts_total);
 			goto suite_summary;
 		}
 	}
@@ -342,6 +372,23 @@ unit_test_suite_runner(struct unit_test_suite *suite)
 		printf(" + TestCase [%2d] : %s %s\n", suite->total,
 				tc.name, status);
 	}
+	FOR_EACH_SUITE_TESTSUITE(i, suite, ts) {
+		ret = unit_test_suite_runner(ts);
+		if (ret == TEST_SUCCESS)
+			sub_ts_succeeded++;
+		else if (ret == TEST_SKIPPED)
+			sub_ts_skipped++;
+		else
+			sub_ts_failed++;
+		sub_ts_total++;
+
+		suite->total += ts->total;
+		suite->succeeded += ts->succeeded;
+		suite->failed += ts->failed;
+		suite->skipped += ts->skipped;
+		suite->unsupported += ts->unsupported;
+		suite->executed += ts->executed;
+	}
 
 	/* Run test suite teardown */
 	if (suite->teardown)
@@ -352,6 +399,23 @@ unit_test_suite_runner(struct unit_test_suite *suite)
 suite_summary:
 	printf(" + ------------------------------------------------------- +\n");
 	printf(" + Test Suite Summary : %s\n", suite->suite_name);
+	printf(" + ------------------------------------------------------- +\n");
+
+	FOR_EACH_SUITE_TESTSUITE(i, suite, ts)
+		printf(" + %s : %d/%d passed, %d/%d skipped, "
+			"%d/%d failed, %d/%d unsupported\n", ts->suite_name,
+			ts->succeeded, ts->total, ts->skipped, ts->total,
+			ts->failed, ts->total, ts->unsupported, ts->total);
+
+	if (suite->unit_test_suites) {
+		printf(" + ------------------------------------------------------- +\n");
+		printf(" + Sub Testsuites Total :     %2d\n", sub_ts_total);
+		printf(" + Sub Testsuites Skipped :   %2d\n", sub_ts_skipped);
+		printf(" + Sub Testsuites Passed :    %2d\n", sub_ts_succeeded);
+		printf(" + Sub Testsuites Failed :    %2d\n", sub_ts_failed);
+		printf(" + ------------------------------------------------------- +\n");
+	}
+
 	printf(" + Tests Total :       %2d\n", suite->total);
 	printf(" + Tests Skipped :     %2d\n", suite->skipped);
 	printf(" + Tests Executed :    %2d\n", suite->executed);
diff --git a/app/test/test.h b/app/test/test.h
index 55f2850fed..f277df7c9d 100644
--- a/app/test/test.h
+++ b/app/test/test.h
@@ -144,6 +144,7 @@ struct unit_test_suite {
 	unsigned int skipped;
 	unsigned int failed;
 	unsigned int unsupported;
+	struct unit_test_suite **unit_test_suites;
 	struct unit_test_case unit_test_cases[];
 };
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH v3 2/7] test: introduce parent testsuite format
  2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 2/7] test: introduce parent testsuite format Ciara Power
@ 2021-04-23 16:32   ` Aaron Conole
  0 siblings, 0 replies; 15+ messages in thread
From: Aaron Conole @ 2021-04-23 16:32 UTC (permalink / raw)
  To: Ciara Power
  Cc: dev, thomas, declan.doherty, gakhil, hemant.agrawal, anoobj,
	ruifeng.wang, asomalap, ajit.khaparde, g.singh

Ciara Power <ciara.power@intel.com> writes:

> The current structure for unit testing only allows for running a
> test suite with nested test cases. This means all test cases for an
> autotest must be in one suite, which is not ideal.
> For example, in some cases we may want to run multiple lists of test
> cases that each require different setup, so should be in separate suites.
>
> The unit test suite struct is modified to hold a pointer to a list of
> sub-testsuite pointers, along with the list of testcases as before.
>
> Signed-off-by: Ciara Power <ciara.power@intel.com>
>
> ---

LGTM

Reviewed-by: Aaron Conole <aconole@redhat.com>


^ permalink raw reply	[flat|nested] 15+ messages in thread

* [dpdk-dev] [PATCH v3 3/7] test/crypto: refactor to use sub-testsuites
  2021-04-23 16:18 [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework Ciara Power
  2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 1/7] app/test: refactor of unit test suite runner Ciara Power
  2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 2/7] test: introduce parent testsuite format Ciara Power
@ 2021-04-23 16:18 ` Ciara Power
  2021-04-30 20:59   ` Doherty, Declan
  2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 4/7] test/crypto: replace unsupported with skipped Ciara Power
                   ` (6 subsequent siblings)
  9 siblings, 1 reply; 15+ messages in thread
From: Ciara Power @ 2021-04-23 16:18 UTC (permalink / raw)
  To: dev
  Cc: thomas, declan.doherty, gakhil, aconole, hemant.agrawal, anoobj,
	ruifeng.wang, asomalap, ajit.khaparde, g.singh, Ciara Power

The existing implementation runs a giant cryptodev testsuite for most
autotests, which in turns runs one setup function regardless of device.

This is now broken down into multiple testsuites,
that are used as sub-testsuites. Each autotest runs a general crypto
parent test suite, to which the sub-testsuites are added.

For example, the AESNI_MB test runs "Cryptodev Unit Test Suite",
which has a setup function only to configure testsuite params.
Creation of vdevs in the setup function is no longer supported,
it is expected the user does this when running the app.
This autotest previously just ran the cryptodev_testsuite,
but now has the smaller sub-testsuites added to the parent suite instead.
The same test cases are being run as before.

The scheduler autotest creates its own parent testsuite with nested
sub-testsuites, rather than using the cryptodev testsuite mentioned above.
This is due to it being more complex in execution,
by requiring setting different modes before running tests.
The scheduler autotest no longer requires the extra test cases to
attach/set mode/detach when running the blockcipher test cases for
each mode. The attach/set mode/detach functionality is now tested in a
sub-testsuite. When running the sub-testsuites for each mode,
the attach/set mode/detach happens in the setup and teardown functions
for that sub-testsuite.

Signed-off-by: Ciara Power <ciara.power@intel.com>

---
v3:
  - Added NULL testcase to each testsuite that has only sub-testsuites,
    and no other testcases.
  - Autotests now run one general crypto testsuite, rather than have
    PMD based parent testsuites.
  - Sub-testsuites now have setup functions which verify the
    capabilities needed for the testcases in that suite are supported.
  - Vdevs are no longer created in the setup functions,
    this must be done by the user when running the test.
v2:
  - Modified sub-testsuites to be added as pointers.
  - Made necessary changes resulting from v2 changes in previous patches.
---
 app/test/test_cryptodev.c | 1797 +++++++++++++++++++++++--------------
 app/test/test_cryptodev.h |   12 +
 2 files changed, 1147 insertions(+), 662 deletions(-)

diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index 32e64e2dd1..4fe30ddb21 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -112,6 +112,10 @@ struct crypto_unittest_params {
 #define ALIGN_POW2_ROUNDUP(num, align) \
 	(((num) + (align) - 1) & ~((align) - 1))
 
+#define ADD_STATIC_TESTSUITE(index, parent_ts, child_ts, num_child_ts)	\
+	for (j = 0; j < num_child_ts; index++, j++)			\
+		parent_ts.unit_test_suites[index] = child_ts[j]
+
 /*
  * Forward declarations.
  */
@@ -490,7 +494,6 @@ testsuite_setup(void)
 	struct crypto_testsuite_params *ts_params = &testsuite_params;
 	struct rte_cryptodev_info info;
 	uint32_t i = 0, nb_devs, dev_id;
-	int ret;
 	uint16_t qp_id;
 
 	memset(ts_params, 0, sizeof(*ts_params));
@@ -536,223 +539,18 @@ testsuite_setup(void)
 		return TEST_FAILED;
 	}
 
-	/* Create an AESNI MB device if required */
-	if (gbl_driver_id == rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD))) {
-		nb_devs = rte_cryptodev_device_count_by_driver(
-				rte_cryptodev_driver_id_get(
-				RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD)));
-		if (nb_devs < 1) {
-			ret = rte_vdev_init(
-				RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD), NULL);
-
-			TEST_ASSERT(ret == 0,
-				"Failed to create instance of"
-				" pmd : %s",
-				RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
-		}
-	}
-
-	/* Create an AESNI GCM device if required */
-	if (gbl_driver_id == rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD))) {
-		nb_devs = rte_cryptodev_device_count_by_driver(
-				rte_cryptodev_driver_id_get(
-				RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD)));
-		if (nb_devs < 1) {
-			TEST_ASSERT_SUCCESS(rte_vdev_init(
-				RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD), NULL),
-				"Failed to create instance of"
-				" pmd : %s",
-				RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD));
-		}
-	}
-
-	/* Create a SNOW 3G device if required */
-	if (gbl_driver_id == rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD))) {
-		nb_devs = rte_cryptodev_device_count_by_driver(
-				rte_cryptodev_driver_id_get(
-				RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD)));
-		if (nb_devs < 1) {
-			TEST_ASSERT_SUCCESS(rte_vdev_init(
-				RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD), NULL),
-				"Failed to create instance of"
-				" pmd : %s",
-				RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD));
-		}
-	}
-
-	/* Create a KASUMI device if required */
-	if (gbl_driver_id == rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_KASUMI_PMD))) {
-		nb_devs = rte_cryptodev_device_count_by_driver(
-				rte_cryptodev_driver_id_get(
-				RTE_STR(CRYPTODEV_NAME_KASUMI_PMD)));
-		if (nb_devs < 1) {
-			TEST_ASSERT_SUCCESS(rte_vdev_init(
-				RTE_STR(CRYPTODEV_NAME_KASUMI_PMD), NULL),
-				"Failed to create instance of"
-				" pmd : %s",
-				RTE_STR(CRYPTODEV_NAME_KASUMI_PMD));
-		}
-	}
-
-	/* Create a ZUC device if required */
-	if (gbl_driver_id == rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_ZUC_PMD))) {
-		nb_devs = rte_cryptodev_device_count_by_driver(
-				rte_cryptodev_driver_id_get(
-				RTE_STR(CRYPTODEV_NAME_ZUC_PMD)));
-		if (nb_devs < 1) {
-			TEST_ASSERT_SUCCESS(rte_vdev_init(
-				RTE_STR(CRYPTODEV_NAME_ZUC_PMD), NULL),
-				"Failed to create instance of"
-				" pmd : %s",
-				RTE_STR(CRYPTODEV_NAME_ZUC_PMD));
-		}
-	}
-
-	/* Create a NULL device if required */
-	if (gbl_driver_id == rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_NULL_PMD))) {
-		nb_devs = rte_cryptodev_device_count_by_driver(
-				rte_cryptodev_driver_id_get(
-				RTE_STR(CRYPTODEV_NAME_NULL_PMD)));
-		if (nb_devs < 1) {
-			ret = rte_vdev_init(
-				RTE_STR(CRYPTODEV_NAME_NULL_PMD), NULL);
-
-			TEST_ASSERT(ret == 0,
-				"Failed to create instance of"
-				" pmd : %s",
-				RTE_STR(CRYPTODEV_NAME_NULL_PMD));
-		}
-	}
-
-	/* Create an OPENSSL device if required */
-	if (gbl_driver_id == rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_OPENSSL_PMD))) {
-		nb_devs = rte_cryptodev_device_count_by_driver(
-				rte_cryptodev_driver_id_get(
-				RTE_STR(CRYPTODEV_NAME_OPENSSL_PMD)));
-		if (nb_devs < 1) {
-			ret = rte_vdev_init(
-				RTE_STR(CRYPTODEV_NAME_OPENSSL_PMD),
-				NULL);
-
-			TEST_ASSERT(ret == 0, "Failed to create "
-				"instance of pmd : %s",
-				RTE_STR(CRYPTODEV_NAME_OPENSSL_PMD));
-		}
-	}
-
-	/* Create a ARMv8 device if required */
-	if (gbl_driver_id == rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_ARMV8_PMD))) {
-		nb_devs = rte_cryptodev_device_count_by_driver(
-				rte_cryptodev_driver_id_get(
-				RTE_STR(CRYPTODEV_NAME_ARMV8_PMD)));
-		if (nb_devs < 1) {
-			ret = rte_vdev_init(
-				RTE_STR(CRYPTODEV_NAME_ARMV8_PMD),
-				NULL);
-
-			TEST_ASSERT(ret == 0, "Failed to create "
-				"instance of pmd : %s",
-				RTE_STR(CRYPTODEV_NAME_ARMV8_PMD));
-		}
-	}
-
-	/* Create a MVSAM device if required */
-	if (gbl_driver_id == rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_MVSAM_PMD))) {
-		nb_devs = rte_cryptodev_device_count_by_driver(
-				rte_cryptodev_driver_id_get(
-				RTE_STR(CRYPTODEV_NAME_MVSAM_PMD)));
-		if (nb_devs < 1) {
-			ret = rte_vdev_init(
-				RTE_STR(CRYPTODEV_NAME_MVSAM_PMD),
-				NULL);
-
-			TEST_ASSERT(ret == 0, "Failed to create "
-				"instance of pmd : %s",
-				RTE_STR(CRYPTODEV_NAME_MVSAM_PMD));
-		}
-	}
-
-	/* Create an CCP device if required */
-	if (gbl_driver_id == rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_CCP_PMD))) {
-		nb_devs = rte_cryptodev_device_count_by_driver(
-				rte_cryptodev_driver_id_get(
-				RTE_STR(CRYPTODEV_NAME_CCP_PMD)));
-		if (nb_devs < 1) {
-			ret = rte_vdev_init(
-				RTE_STR(CRYPTODEV_NAME_CCP_PMD),
-				NULL);
-
-			TEST_ASSERT(ret == 0, "Failed to create "
-				"instance of pmd : %s",
-				RTE_STR(CRYPTODEV_NAME_CCP_PMD));
-		}
-	}
-
-#ifdef RTE_CRYPTO_SCHEDULER
-	char vdev_args[VDEV_ARGS_SIZE] = {""};
-	char temp_str[VDEV_ARGS_SIZE] = {"mode=multi-core,"
-		"ordering=enable,name=cryptodev_test_scheduler,corelist="};
-	uint16_t worker_core_count = 0;
-	uint16_t socket_id = 0;
-
-	if (gbl_driver_id == rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD))) {
-
-		/* Identify the Worker Cores
-		 * Use 2 worker cores for the device args
-		 */
-		RTE_LCORE_FOREACH_WORKER(i) {
-			if (worker_core_count > 1)
-				break;
-			snprintf(vdev_args, sizeof(vdev_args),
-					"%s%d", temp_str, i);
-			strcpy(temp_str, vdev_args);
-			strlcat(temp_str, ";", sizeof(temp_str));
-			worker_core_count++;
-			socket_id = rte_lcore_to_socket_id(i);
-		}
-		if (worker_core_count != 2) {
-			RTE_LOG(ERR, USER1,
-				"Cryptodev scheduler test require at least "
-				"two worker cores to run. "
-				"Please use the correct coremask.\n");
-			return TEST_FAILED;
-		}
-		strcpy(temp_str, vdev_args);
-		snprintf(vdev_args, sizeof(vdev_args), "%s,socket_id=%d",
-				temp_str, socket_id);
-		RTE_LOG(DEBUG, USER1, "vdev_args: %s\n", vdev_args);
-		nb_devs = rte_cryptodev_device_count_by_driver(
-				rte_cryptodev_driver_id_get(
-				RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD)));
-		if (nb_devs < 1) {
-			ret = rte_vdev_init(
-				RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD),
-					vdev_args);
-			TEST_ASSERT(ret == 0,
-				"Failed to create instance %u of"
-				" pmd : %s",
-				i, RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD));
-		}
-	}
-#endif /* RTE_CRYPTO_SCHEDULER */
-
 	nb_devs = rte_cryptodev_count();
 	if (nb_devs < 1) {
 		RTE_LOG(WARNING, USER1, "No crypto devices found?\n");
 		return TEST_SKIPPED;
 	}
 
+	if (rte_cryptodev_device_count_by_driver(gbl_driver_id) < 1) {
+		RTE_LOG(WARNING, USER1, "No %s devices found?\n",
+				rte_cryptodev_driver_name_get(gbl_driver_id));
+		return TEST_SKIPPED;
+	}
+
 	/* Create list of valid crypto devs */
 	for (i = 0; i < nb_devs; i++) {
 		rte_cryptodev_info_get(i, &info);
@@ -865,125 +663,733 @@ testsuite_teardown(void)
 }
 
 static int
-dev_configure_and_start(uint64_t ff_disable)
+check_capabilities_supported(enum rte_crypto_sym_xform_type type,
+		const int *algs, uint16_t num_algs)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
-	struct crypto_unittest_params *ut_params = &unittest_params;
-
-	uint16_t qp_id;
-
-	/* Clear unit test parameters before running test */
-	memset(ut_params, 0, sizeof(*ut_params));
-
-	/* Reconfigure device to default parameters */
-	ts_params->conf.socket_id = SOCKET_ID_ANY;
-	ts_params->conf.ff_disable = ff_disable;
-	ts_params->qp_conf.nb_descriptors = MAX_NUM_OPS_INFLIGHT;
-	ts_params->qp_conf.mp_session = ts_params->session_mpool;
-	ts_params->qp_conf.mp_session_private = ts_params->session_priv_mpool;
-
-	TEST_ASSERT_SUCCESS(rte_cryptodev_configure(ts_params->valid_devs[0],
-			&ts_params->conf),
-			"Failed to configure cryptodev %u",
-			ts_params->valid_devs[0]);
+	uint8_t dev_id = testsuite_params.valid_devs[0];
+	bool some_alg_supported = FALSE;
+	uint16_t i;
 
-	for (qp_id = 0; qp_id < ts_params->conf.nb_queue_pairs ; qp_id++) {
-		TEST_ASSERT_SUCCESS(rte_cryptodev_queue_pair_setup(
-			ts_params->valid_devs[0], qp_id,
-			&ts_params->qp_conf,
-			rte_cryptodev_socket_id(ts_params->valid_devs[0])),
-			"Failed to setup queue pair %u on cryptodev %u",
-			qp_id, ts_params->valid_devs[0]);
+	for (i = 0; i < num_algs && !some_alg_supported; i++) {
+		struct rte_cryptodev_sym_capability_idx alg = {
+			type, {algs[i]}
+		};
+		if (rte_cryptodev_sym_capability_get(dev_id,
+				&alg) != NULL)
+			some_alg_supported = TRUE;
 	}
+	if (!some_alg_supported)
+		return TEST_SKIPPED;
 
+	return 0;
+}
 
-	rte_cryptodev_stats_reset(ts_params->valid_devs[0]);
-
-	/* Start the device */
-	TEST_ASSERT_SUCCESS(rte_cryptodev_start(ts_params->valid_devs[0]),
-			"Failed to start cryptodev %u",
-			ts_params->valid_devs[0]);
-
-	return TEST_SUCCESS;
+int
+check_cipher_capabilities_supported(const enum rte_crypto_cipher_algorithm *ciphers,
+		uint16_t num_ciphers)
+{
+	return check_capabilities_supported(RTE_CRYPTO_SYM_XFORM_CIPHER,
+			(const int *) ciphers, num_ciphers);
 }
 
-static int
-ut_setup(void)
+int
+check_auth_capabilities_supported(const enum rte_crypto_auth_algorithm *auths,
+		uint16_t num_auths)
 {
-	/* Configure and start the device with security feature disabled */
-	return dev_configure_and_start(RTE_CRYPTODEV_FF_SECURITY);
+	return check_capabilities_supported(RTE_CRYPTO_SYM_XFORM_AUTH,
+			(const int *) auths, num_auths);
 }
 
-static int
-ut_setup_security(void)
+int
+check_aead_capabilities_supported(const enum rte_crypto_aead_algorithm *aeads,
+		uint16_t num_aeads)
 {
-	/* Configure and start the device with no features disabled */
-	return dev_configure_and_start(0);
+	return check_capabilities_supported(RTE_CRYPTO_SYM_XFORM_AEAD,
+			(const int *) aeads, num_aeads);
 }
 
-static void
-ut_teardown(void)
+static int
+null_testsuite_setup(void)
 {
 	struct crypto_testsuite_params *ts_params = &testsuite_params;
-	struct crypto_unittest_params *ut_params = &unittest_params;
-	struct rte_cryptodev_stats stats;
-
-	/* free crypto session structure */
-#ifdef RTE_LIB_SECURITY
-	if (ut_params->type == RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL) {
-		if (ut_params->sec_session) {
-			rte_security_session_destroy(rte_cryptodev_get_sec_ctx
-						(ts_params->valid_devs[0]),
-						ut_params->sec_session);
-			ut_params->sec_session = NULL;
-		}
-	} else
-#endif
-	{
-		if (ut_params->sess) {
-			rte_cryptodev_sym_session_clear(
-					ts_params->valid_devs[0],
-					ut_params->sess);
-			rte_cryptodev_sym_session_free(ut_params->sess);
-			ut_params->sess = NULL;
-		}
-	}
+	uint8_t dev_id = ts_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	const enum rte_crypto_cipher_algorithm ciphers[] = {
+		RTE_CRYPTO_CIPHER_NULL
+	};
+	const enum rte_crypto_auth_algorithm auths[] = {
+		RTE_CRYPTO_AUTH_NULL
+	};
 
-	/* free crypto operation structure */
-	if (ut_params->op)
-		rte_crypto_op_free(ut_params->op);
+	rte_cryptodev_info_get(dev_id, &dev_info);
 
-	/*
-	 * free mbuf - both obuf and ibuf are usually the same,
-	 * so check if they point at the same address is necessary,
-	 * to avoid freeing the mbuf twice.
-	 */
-	if (ut_params->obuf) {
-		rte_pktmbuf_free(ut_params->obuf);
-		if (ut_params->ibuf == ut_params->obuf)
-			ut_params->ibuf = 0;
-		ut_params->obuf = 0;
-	}
-	if (ut_params->ibuf) {
-		rte_pktmbuf_free(ut_params->ibuf);
-		ut_params->ibuf = 0;
+	if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO)) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for NULL "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
 	}
 
-	if (ts_params->mbuf_pool != NULL)
-		RTE_LOG(DEBUG, USER1, "CRYPTO_MBUFPOOL count %u\n",
-			rte_mempool_avail_count(ts_params->mbuf_pool));
-
-	rte_cryptodev_stats_get(ts_params->valid_devs[0], &stats);
+	if (check_cipher_capabilities_supported(ciphers, RTE_DIM(ciphers)) != 0
+			&& check_auth_capabilities_supported(auths,
+			RTE_DIM(auths)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for NULL "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
 
-	/* Stop the device */
-	rte_cryptodev_stop(ts_params->valid_devs[0]);
+	return 0;
 }
 
 static int
-test_device_configure_invalid_dev_id(void)
+crypto_gen_testsuite_setup(void)
 {
 	struct crypto_testsuite_params *ts_params = &testsuite_params;
-	uint16_t dev_id, num_devs = 0;
+	uint8_t dev_id = ts_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+
+	if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO)) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for Crypto Gen "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+#ifdef RTE_LIB_SECURITY
+static int
+pdcp_proto_testsuite_setup(void)
+{
+	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	uint8_t dev_id = ts_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	const enum rte_crypto_cipher_algorithm ciphers[] = {
+		RTE_CRYPTO_CIPHER_NULL,
+		RTE_CRYPTO_CIPHER_AES_CTR,
+		RTE_CRYPTO_CIPHER_ZUC_EEA3,
+		RTE_CRYPTO_CIPHER_SNOW3G_UEA2
+	};
+	const enum rte_crypto_auth_algorithm auths[] = {
+		RTE_CRYPTO_AUTH_NULL,
+		RTE_CRYPTO_AUTH_SNOW3G_UIA2,
+		RTE_CRYPTO_AUTH_AES_CMAC,
+		RTE_CRYPTO_AUTH_ZUC_EIA3
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+
+	if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			!(dev_info.feature_flags &
+			RTE_CRYPTODEV_FF_SECURITY)) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for PDCP Proto "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_cipher_capabilities_supported(ciphers, RTE_DIM(ciphers)) != 0
+			&& check_auth_capabilities_supported(auths,
+			RTE_DIM(auths)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for PDCP Proto "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+docsis_proto_testsuite_setup(void)
+{
+	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	uint8_t dev_id = ts_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	const enum rte_crypto_cipher_algorithm ciphers[] = {
+		RTE_CRYPTO_CIPHER_AES_DOCSISBPI
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+
+	if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			!(dev_info.feature_flags &
+			RTE_CRYPTODEV_FF_SECURITY)) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for Docsis "
+				"Proto testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_cipher_capabilities_supported(ciphers, RTE_DIM(ciphers)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for Docsis Proto "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+#endif
+
+static int
+aes_ccm_auth_testsuite_setup(void)
+{
+	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	uint8_t dev_id = ts_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	const enum rte_crypto_aead_algorithm aeads[] = {
+		RTE_CRYPTO_AEAD_AES_CCM
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+
+	if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for AES CCM "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_aead_capabilities_supported(aeads, RTE_DIM(aeads)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for AES CCM "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+aes_gcm_auth_testsuite_setup(void)
+{
+	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	uint8_t dev_id = ts_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	const enum rte_crypto_aead_algorithm aeads[] = {
+		RTE_CRYPTO_AEAD_AES_GCM
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+
+	if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO)) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for AES GCM "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_aead_capabilities_supported(aeads, RTE_DIM(aeads)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for AES GCM "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+aes_gmac_auth_testsuite_setup(void)
+{
+	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	uint8_t dev_id = ts_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	const enum rte_crypto_auth_algorithm auths[] = {
+		RTE_CRYPTO_AUTH_AES_GMAC
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+
+	if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for AES GMAC "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_auth_capabilities_supported(auths, RTE_DIM(auths)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for AES GMAC "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+chacha20_poly1305_testsuite_setup(void)
+{
+	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	uint8_t dev_id = ts_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	const enum rte_crypto_aead_algorithm aeads[] = {
+		RTE_CRYPTO_AEAD_CHACHA20_POLY1305
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+
+	if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for "
+				"Chacha20-Poly1305 testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_aead_capabilities_supported(aeads, RTE_DIM(aeads)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for "
+				"Chacha20-Poly1305 testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+snow3g_testsuite_setup(void)
+{
+	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	uint8_t dev_id = ts_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	const enum rte_crypto_cipher_algorithm ciphers[] = {
+		RTE_CRYPTO_CIPHER_SNOW3G_UEA2
+
+	};
+	const enum rte_crypto_auth_algorithm auths[] = {
+		RTE_CRYPTO_AUTH_SNOW3G_UIA2
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+
+	if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO)) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for Snow3G "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_cipher_capabilities_supported(ciphers, RTE_DIM(ciphers)) != 0
+			&& check_auth_capabilities_supported(auths,
+			RTE_DIM(auths)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for Snow3G "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+zuc_testsuite_setup(void)
+{
+	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	uint8_t dev_id = ts_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	const enum rte_crypto_cipher_algorithm ciphers[] = {
+		RTE_CRYPTO_CIPHER_ZUC_EEA3
+	};
+	const enum rte_crypto_auth_algorithm auths[] = {
+		RTE_CRYPTO_AUTH_ZUC_EIA3
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+
+	if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO)) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for ZUC "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_cipher_capabilities_supported(ciphers, RTE_DIM(ciphers)) != 0
+			&& check_auth_capabilities_supported(auths,
+			RTE_DIM(auths)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for ZUC "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+hmac_md5_auth_testsuite_setup(void)
+{
+	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	uint8_t dev_id = ts_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	const enum rte_crypto_auth_algorithm auths[] = {
+		RTE_CRYPTO_AUTH_MD5_HMAC
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+
+	if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for HMAC MD5 "
+				"Auth testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_auth_capabilities_supported(auths, RTE_DIM(auths)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for HMAC MD5 "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+kasumi_testsuite_setup(void)
+{
+	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	uint8_t dev_id = ts_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	const enum rte_crypto_cipher_algorithm ciphers[] = {
+		RTE_CRYPTO_CIPHER_KASUMI_F8
+	};
+	const enum rte_crypto_auth_algorithm auths[] = {
+		RTE_CRYPTO_AUTH_KASUMI_F9
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+
+	if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for Kasumi "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_cipher_capabilities_supported(ciphers, RTE_DIM(ciphers)) != 0
+			&& check_auth_capabilities_supported(auths,
+			RTE_DIM(auths)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for Kasumi "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+negative_aes_gcm_testsuite_setup(void)
+{
+	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	uint8_t dev_id = ts_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	const enum rte_crypto_aead_algorithm aeads[] = {
+		RTE_CRYPTO_AEAD_AES_GCM
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+
+	if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for Negative "
+				"AES GCM testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_aead_capabilities_supported(aeads, RTE_DIM(aeads)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for Negative "
+				"AES GCM testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+negative_aes_gmac_testsuite_setup(void)
+{
+	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	uint8_t dev_id = ts_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	const enum rte_crypto_auth_algorithm auths[] = {
+		RTE_CRYPTO_AUTH_AES_GMAC
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+
+	if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for Negative "
+				"AES GMAC testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_auth_capabilities_supported(auths, RTE_DIM(auths)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for Negative "
+				"AES GMAC testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+mixed_cipher_hash_testsuite_setup(void)
+{
+	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	uint8_t dev_id = ts_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	uint64_t feat_flags;
+	const enum rte_crypto_cipher_algorithm ciphers[] = {
+		RTE_CRYPTO_CIPHER_NULL,
+		RTE_CRYPTO_CIPHER_AES_CTR,
+		RTE_CRYPTO_CIPHER_ZUC_EEA3,
+		RTE_CRYPTO_CIPHER_SNOW3G_UEA2
+	};
+	const enum rte_crypto_auth_algorithm auths[] = {
+		RTE_CRYPTO_AUTH_NULL,
+		RTE_CRYPTO_AUTH_SNOW3G_UIA2,
+		RTE_CRYPTO_AUTH_AES_CMAC,
+		RTE_CRYPTO_AUTH_ZUC_EIA3
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+	feat_flags = dev_info.feature_flags;
+
+	if (!(feat_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			(global_api_test_type == CRYPTODEV_RAW_API_TEST)) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for Mixed "
+				"Cipher Hash testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_cipher_capabilities_supported(ciphers, RTE_DIM(ciphers)) != 0
+			&& check_auth_capabilities_supported(auths,
+			RTE_DIM(auths)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for Mixed "
+				"Cipher Hash testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+esn_testsuite_setup(void)
+{
+	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	uint8_t dev_id = ts_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	const enum rte_crypto_cipher_algorithm ciphers[] = {
+		RTE_CRYPTO_CIPHER_AES_CBC
+	};
+	const enum rte_crypto_auth_algorithm auths[] = {
+		RTE_CRYPTO_AUTH_SHA1_HMAC
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+
+	if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for ESN "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_cipher_capabilities_supported(ciphers, RTE_DIM(ciphers)) != 0
+			&& check_auth_capabilities_supported(auths,
+			RTE_DIM(auths)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for ESN "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+multi_session_testsuite_setup(void)
+{
+	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	uint8_t dev_id = ts_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	const enum rte_crypto_cipher_algorithm ciphers[] = {
+		RTE_CRYPTO_CIPHER_AES_CBC
+	};
+	const enum rte_crypto_auth_algorithm auths[] = {
+		RTE_CRYPTO_AUTH_SHA512_HMAC
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+
+	if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO)) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for Multi "
+				"Session testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_cipher_capabilities_supported(ciphers, RTE_DIM(ciphers)) != 0
+			&& check_auth_capabilities_supported(auths,
+			RTE_DIM(auths)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for Multi "
+				"Session testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+negative_hmac_sha1_testsuite_setup(void)
+{
+	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	uint8_t dev_id = ts_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	const enum rte_crypto_cipher_algorithm ciphers[] = {
+		RTE_CRYPTO_CIPHER_AES_CBC
+	};
+	const enum rte_crypto_auth_algorithm auths[] = {
+		RTE_CRYPTO_AUTH_SHA1_HMAC
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+
+	if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			!(dev_info.feature_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for Negative "
+				"HMAC SHA1 testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_cipher_capabilities_supported(ciphers, RTE_DIM(ciphers)) != 0
+			&& check_auth_capabilities_supported(auths,
+			RTE_DIM(auths)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for Negative "
+				"HMAC SHA1 testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+dev_configure_and_start(uint64_t ff_disable)
+{
+	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_unittest_params *ut_params = &unittest_params;
+
+	uint16_t qp_id;
+
+	/* Clear unit test parameters before running test */
+	memset(ut_params, 0, sizeof(*ut_params));
+
+	/* Reconfigure device to default parameters */
+	ts_params->conf.socket_id = SOCKET_ID_ANY;
+	ts_params->conf.ff_disable = ff_disable;
+	ts_params->qp_conf.nb_descriptors = MAX_NUM_OPS_INFLIGHT;
+	ts_params->qp_conf.mp_session = ts_params->session_mpool;
+	ts_params->qp_conf.mp_session_private = ts_params->session_priv_mpool;
+
+	TEST_ASSERT_SUCCESS(rte_cryptodev_configure(ts_params->valid_devs[0],
+			&ts_params->conf),
+			"Failed to configure cryptodev %u",
+			ts_params->valid_devs[0]);
+
+	for (qp_id = 0; qp_id < ts_params->conf.nb_queue_pairs ; qp_id++) {
+		TEST_ASSERT_SUCCESS(rte_cryptodev_queue_pair_setup(
+			ts_params->valid_devs[0], qp_id,
+			&ts_params->qp_conf,
+			rte_cryptodev_socket_id(ts_params->valid_devs[0])),
+			"Failed to setup queue pair %u on cryptodev %u",
+			qp_id, ts_params->valid_devs[0]);
+	}
+
+
+	rte_cryptodev_stats_reset(ts_params->valid_devs[0]);
+
+	/* Start the device */
+	TEST_ASSERT_SUCCESS(rte_cryptodev_start(ts_params->valid_devs[0]),
+			"Failed to start cryptodev %u",
+			ts_params->valid_devs[0]);
+
+	return TEST_SUCCESS;
+}
+
+static int
+ut_setup(void)
+{
+	/* Configure and start the device with security feature disabled */
+	return dev_configure_and_start(RTE_CRYPTODEV_FF_SECURITY);
+}
+
+static int
+ut_setup_security(void)
+{
+	/* Configure and start the device with no features disabled */
+	return dev_configure_and_start(0);
+}
+
+static void
+ut_teardown(void)
+{
+	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_unittest_params *ut_params = &unittest_params;
+	struct rte_cryptodev_stats stats;
+
+	/* free crypto session structure */
+#ifdef RTE_LIB_SECURITY
+	if (ut_params->type == RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL) {
+		if (ut_params->sec_session) {
+			rte_security_session_destroy(rte_cryptodev_get_sec_ctx
+						(ts_params->valid_devs[0]),
+						ut_params->sec_session);
+			ut_params->sec_session = NULL;
+		}
+	} else
+#endif
+	{
+		if (ut_params->sess) {
+			rte_cryptodev_sym_session_clear(
+					ts_params->valid_devs[0],
+					ut_params->sess);
+			rte_cryptodev_sym_session_free(ut_params->sess);
+			ut_params->sess = NULL;
+		}
+	}
+
+	/* free crypto operation structure */
+	if (ut_params->op)
+		rte_crypto_op_free(ut_params->op);
+
+	/*
+	 * free mbuf - both obuf and ibuf are usually the same,
+	 * so check if they point at the same address is necessary,
+	 * to avoid freeing the mbuf twice.
+	 */
+	if (ut_params->obuf) {
+		rte_pktmbuf_free(ut_params->obuf);
+		if (ut_params->ibuf == ut_params->obuf)
+			ut_params->ibuf = 0;
+		ut_params->obuf = 0;
+	}
+	if (ut_params->ibuf) {
+		rte_pktmbuf_free(ut_params->ibuf);
+		ut_params->ibuf = 0;
+	}
+
+	if (ts_params->mbuf_pool != NULL)
+		RTE_LOG(DEBUG, USER1, "CRYPTO_MBUFPOOL count %u\n",
+			rte_mempool_avail_count(ts_params->mbuf_pool));
+
+	rte_cryptodev_stats_get(ts_params->valid_devs[0], &stats);
+
+	/* Stop the device */
+	rte_cryptodev_stop(ts_params->valid_devs[0]);
+}
+
+static int
+test_device_configure_invalid_dev_id(void)
+{
+	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	uint16_t dev_id, num_devs = 0;
 
 	TEST_ASSERT((num_devs = rte_cryptodev_count()) >= 1,
 			"Need at least %d devices for test", 1);
@@ -13068,6 +13474,59 @@ test_chacha20_poly1305_decrypt_test_case_rfc8439(void)
 /* global AESNI worker IDs for the scheduler test */
 uint8_t aesni_ids[2];
 
+static int
+scheduler_testsuite_setup(void)
+{
+	uint32_t i = 0;
+	int32_t nb_devs, ret;
+	char vdev_args[VDEV_ARGS_SIZE] = {""};
+	char temp_str[VDEV_ARGS_SIZE] = {"mode=multi-core,"
+		"ordering=enable,name=cryptodev_test_scheduler,corelist="};
+	uint16_t worker_core_count = 0;
+	uint16_t socket_id = 0;
+
+	if (gbl_driver_id == rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD))) {
+
+		/* Identify the Worker Cores
+		 * Use 2 worker cores for the device args
+		 */
+		RTE_LCORE_FOREACH_WORKER(i) {
+			if (worker_core_count > 1)
+				break;
+			snprintf(vdev_args, sizeof(vdev_args),
+					"%s%d", temp_str, i);
+			strcpy(temp_str, vdev_args);
+			strlcat(temp_str, ";", sizeof(temp_str));
+			worker_core_count++;
+			socket_id = rte_lcore_to_socket_id(i);
+		}
+		if (worker_core_count != 2) {
+			RTE_LOG(ERR, USER1,
+				"Cryptodev scheduler test require at least "
+				"two worker cores to run. "
+				"Please use the correct coremask.\n");
+			return TEST_FAILED;
+		}
+		strcpy(temp_str, vdev_args);
+		snprintf(vdev_args, sizeof(vdev_args), "%s,socket_id=%d",
+				temp_str, socket_id);
+		RTE_LOG(DEBUG, USER1, "vdev_args: %s\n", vdev_args);
+		nb_devs = rte_cryptodev_device_count_by_driver(
+				rte_cryptodev_driver_id_get(
+				RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD)));
+		if (nb_devs < 1) {
+			ret = rte_vdev_init(
+				RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD),
+					vdev_args);
+			TEST_ASSERT(ret == 0,
+				"Failed to create instance %u of pmd : %s",
+				i, RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD));
+		}
+	}
+	return testsuite_setup();
+}
+
 static int
 test_scheduler_attach_slave_op(void)
 {
@@ -13242,53 +13701,86 @@ test_scheduler_mode_pkt_size_distr_op(void)
 	return 0;
 }
 
-static struct unit_test_suite cryptodev_scheduler_testsuite  = {
-	.suite_name = "Crypto Device Scheduler Unit Test Suite",
-	.setup = testsuite_setup,
-	.teardown = testsuite_teardown,
-	.unit_test_cases = {
-		/* Multi Core */
-		TEST_CASE_ST(NULL, NULL, test_scheduler_attach_slave_op),
-		TEST_CASE_ST(NULL, NULL, test_scheduler_mode_multicore_op),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_cipheronly_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_authonly_all),
-		TEST_CASE_ST(NULL, NULL, test_scheduler_detach_slave_op),
+static int
+scheduler_multicore_testsuite_setup(void)
+{
+	if (test_scheduler_attach_slave_op() < 0)
+		return TEST_SKIPPED;
+	if (test_scheduler_mode_op(CDEV_SCHED_MODE_MULTICORE) < 0)
+		return TEST_SKIPPED;
+	return 0;
+}
 
-		/* Round Robin */
-		TEST_CASE_ST(NULL, NULL, test_scheduler_attach_slave_op),
-		TEST_CASE_ST(NULL, NULL, test_scheduler_mode_roundrobin_op),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_cipheronly_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_authonly_all),
-		TEST_CASE_ST(NULL, NULL, test_scheduler_detach_slave_op),
+static int
+scheduler_roundrobin_testsuite_setup(void)
+{
+	if (test_scheduler_attach_slave_op() < 0)
+		return TEST_SKIPPED;
+	if (test_scheduler_mode_op(CDEV_SCHED_MODE_ROUNDROBIN) < 0)
+		return TEST_SKIPPED;
+	return 0;
+}
 
-		/* Fail over */
-		TEST_CASE_ST(NULL, NULL, test_scheduler_attach_slave_op),
-		TEST_CASE_ST(NULL, NULL, test_scheduler_mode_failover_op),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_cipheronly_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_authonly_all),
-		TEST_CASE_ST(NULL, NULL, test_scheduler_detach_slave_op),
+static int
+scheduler_failover_testsuite_setup(void)
+{
+	if (test_scheduler_attach_slave_op() < 0)
+		return TEST_SKIPPED;
+	if (test_scheduler_mode_op(CDEV_SCHED_MODE_FAILOVER) < 0)
+		return TEST_SKIPPED;
+	return 0;
+}
 
-		/* PKT SIZE */
-		TEST_CASE_ST(NULL, NULL, test_scheduler_attach_slave_op),
-		TEST_CASE_ST(NULL, NULL, test_scheduler_mode_pkt_size_distr_op),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_cipheronly_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_authonly_all),
-		TEST_CASE_ST(NULL, NULL, test_scheduler_detach_slave_op),
+static int
+scheduler_pkt_size_distr_testsuite_setup(void)
+{
+	if (test_scheduler_attach_slave_op() < 0)
+		return TEST_SKIPPED;
+	if (test_scheduler_mode_op(CDEV_SCHED_MODE_PKT_SIZE_DISTR) < 0)
+		return TEST_SKIPPED;
+	return 0;
+}
+
+static void
+scheduler_mode_testsuite_teardown(void)
+{
+	test_scheduler_detach_slave_op();
+}
+
+#endif /* RTE_CRYPTO_SCHEDULER */
+
+static struct unit_test_suite end_testsuite = {
+	.suite_name = NULL,
+	.setup = NULL,
+	.teardown = NULL,
+	.unit_test_suites = NULL
+};
 
+#ifdef RTE_LIB_SECURITY
+static struct unit_test_suite pdcp_proto_testsuite  = {
+	.suite_name = "PDCP Proto Unit Test Suite",
+	.setup = pdcp_proto_testsuite_setup,
+	.unit_test_cases = {
+		TEST_CASE_ST(ut_setup_security, ut_teardown,
+			test_PDCP_PROTO_all),
 		TEST_CASES_END() /**< NULL terminate unit test array */
 	}
 };
 
-#endif /* RTE_CRYPTO_SCHEDULER */
+static struct unit_test_suite docsis_proto_testsuite  = {
+	.suite_name = "Docsis Proto Unit Test Suite",
+	.setup = docsis_proto_testsuite_setup,
+	.unit_test_cases = {
+		TEST_CASE_ST(ut_setup_security, ut_teardown,
+			test_DOCSIS_PROTO_all),
+		TEST_CASES_END() /**< NULL terminate unit test array */
+	}
+};
+#endif
 
-static struct unit_test_suite cryptodev_testsuite  = {
-	.suite_name = "Crypto Unit Test Suite",
-	.setup = testsuite_setup,
-	.teardown = testsuite_teardown,
+static struct unit_test_suite cryptodev_gen_testsuite  = {
+	.suite_name = "Crypto General Unit Test Suite",
+	.setup = crypto_gen_testsuite_setup,
 	.unit_test_cases = {
 		TEST_CASE_ST(ut_setup, ut_teardown,
 				test_device_configure_invalid_dev_id),
@@ -13296,14 +13788,6 @@ static struct unit_test_suite cryptodev_testsuite  = {
 				test_queue_pair_descriptor_setup),
 		TEST_CASE_ST(ut_setup, ut_teardown,
 				test_device_configure_invalid_queue_pair_ids),
-		TEST_CASE_ST(ut_setup, ut_teardown,
-				test_multi_session),
-		TEST_CASE_ST(ut_setup, ut_teardown,
-				test_multi_session_random_usage),
-
-		TEST_CASE_ST(ut_setup, ut_teardown,
-			test_null_invalid_operation),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_null_burst_operation),
 		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
 		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_cipheronly_all),
 		TEST_CASE_ST(ut_setup, ut_teardown, test_3DES_chain_all),
@@ -13313,8 +13797,58 @@ static struct unit_test_suite cryptodev_testsuite  = {
 		TEST_CASE_ST(ut_setup, ut_teardown, test_DES_docsis_all),
 		TEST_CASE_ST(ut_setup, ut_teardown, test_authonly_all),
 		TEST_CASE_ST(ut_setup, ut_teardown, test_stats),
+		TEST_CASE_ST(ut_setup, ut_teardown, test_enq_callback_setup),
+		TEST_CASE_ST(ut_setup, ut_teardown, test_deq_callback_setup),
+		TEST_CASES_END() /**< NULL terminate unit test array */
+	}
+};
+
+static struct unit_test_suite cryptodev_negative_hmac_sha1_testsuite = {
+	.suite_name = "Negative HMAC SHA1 Unit Test Suite",
+	.setup = negative_hmac_sha1_testsuite_setup,
+	.unit_test_cases = {
+		/** Negative tests */
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			authentication_verify_HMAC_SHA1_fail_data_corrupt),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			authentication_verify_HMAC_SHA1_fail_tag_corrupt),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			auth_decryption_AES128CBC_HMAC_SHA1_fail_data_corrupt),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			auth_decryption_AES128CBC_HMAC_SHA1_fail_tag_corrupt),
+
+		TEST_CASES_END() /**< NULL terminate unit test array */
+	}
+};
+
+static struct unit_test_suite cryptodev_multi_session_testsuite = {
+	.suite_name = "Multi Session Unit Test Suite",
+	.setup = multi_session_testsuite_setup,
+	.unit_test_cases = {
+		TEST_CASE_ST(ut_setup, ut_teardown, test_multi_session),
+		TEST_CASE_ST(ut_setup, ut_teardown,
+				test_multi_session_random_usage),
+
+		TEST_CASES_END() /**< NULL terminate unit test array */
+	}
+};
+
+static struct unit_test_suite cryptodev_null_testsuite  = {
+	.suite_name = "NULL Test Suite",
+	.setup = null_testsuite_setup,
+	.unit_test_cases = {
+		TEST_CASE_ST(ut_setup, ut_teardown,
+			test_null_invalid_operation),
+		TEST_CASE_ST(ut_setup, ut_teardown, test_null_burst_operation),
+		TEST_CASES_END()
+	}
+};
 
-		/** AES CCM Authenticated Encryption 128 bits key */
+static struct unit_test_suite cryptodev_aes_ccm_auth_testsuite  = {
+	.suite_name = "AES CCM Authenticated Test Suite",
+	.setup = aes_ccm_auth_testsuite_setup,
+	.unit_test_cases = {
+		/** AES CCM Authenticated Encryption 128 bits key*/
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_AES_CCM_authenticated_encryption_test_case_128_1),
 		TEST_CASE_ST(ut_setup, ut_teardown,
@@ -13361,7 +13895,14 @@ static struct unit_test_suite cryptodev_testsuite  = {
 			test_AES_CCM_authenticated_decryption_test_case_256_2),
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_AES_CCM_authenticated_decryption_test_case_256_3),
+		TEST_CASES_END()
+	}
+};
 
+static struct unit_test_suite cryptodev_aes_gcm_auth_testsuite  = {
+	.suite_name = "AES GCM Authenticated Test Suite",
+	.setup = aes_gcm_auth_testsuite_setup,
+	.unit_test_cases = {
 		/** AES GCM Authenticated Encryption */
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_AES_GCM_auth_encrypt_SGL_in_place_1500B),
@@ -13498,7 +14039,14 @@ static struct unit_test_suite cryptodev_testsuite  = {
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_AES_GCM_authenticated_decryption_sessionless_test_case_1),
 
-		/** AES GMAC Authentication */
+		TEST_CASES_END()
+	}
+};
+
+static struct unit_test_suite cryptodev_aes_gmac_auth_testsuite  = {
+	.suite_name = "AES GMAC Authentication Test Suite",
+	.setup = aes_gmac_auth_testsuite_setup,
+	.unit_test_cases = {
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_AES_GMAC_authentication_test_case_1),
 		TEST_CASE_ST(ut_setup, ut_teardown,
@@ -13524,11 +14072,26 @@ static struct unit_test_suite cryptodev_testsuite  = {
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_AES_GMAC_authentication_SGL_2047B),
 
-		/** Chacha20-Poly1305 */
+		TEST_CASES_END()
+	}
+};
+
+static struct unit_test_suite cryptodev_chacha20_poly1305_testsuite  = {
+	.suite_name = "Chacha20-Poly1305 Test Suite",
+	.setup = chacha20_poly1305_testsuite_setup,
+	.unit_test_cases = {
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_chacha20_poly1305_encrypt_test_case_rfc8439),
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_chacha20_poly1305_decrypt_test_case_rfc8439),
+		TEST_CASES_END()
+	}
+};
+
+static struct unit_test_suite cryptodev_snow3g_testsuite  = {
+	.suite_name = "SNOW 3G Test Suite",
+	.setup = snow3g_testsuite_setup,
+	.unit_test_cases = {
 		/** SNOW 3G encrypt only (UEA2) */
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_snow3g_encryption_test_case_1),
@@ -13609,6 +14172,7 @@ static struct unit_test_suite cryptodev_testsuite  = {
 			test_snow3g_hash_generate_test_case_2),
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_snow3g_hash_generate_test_case_3),
+
 		/* Tests with buffers which length is not byte-aligned */
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_snow3g_hash_generate_test_case_4),
@@ -13622,6 +14186,7 @@ static struct unit_test_suite cryptodev_testsuite  = {
 			test_snow3g_hash_verify_test_case_2),
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_snow3g_hash_verify_test_case_3),
+
 		/* Tests with buffers which length is not byte-aligned */
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_snow3g_hash_verify_test_case_4),
@@ -13633,7 +14198,14 @@ static struct unit_test_suite cryptodev_testsuite  = {
 			test_snow3g_cipher_auth_test_case_1),
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_snow3g_auth_cipher_with_digest_test_case_1),
+		TEST_CASES_END()
+	}
+};
 
+static struct unit_test_suite cryptodev_zuc_testsuite  = {
+	.suite_name = "ZUC Test Suite",
+	.setup = zuc_testsuite_setup,
+	.unit_test_cases = {
 		/** ZUC encrypt only (EEA3) */
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_zuc_encryption_test_case_1),
@@ -13691,8 +14263,14 @@ static struct unit_test_suite cryptodev_testsuite  = {
 			test_zuc_auth_cipher_verify_test_case_1_sgl),
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_zuc_auth_cipher_verify_test_case_1_oop_sgl),
+		TEST_CASES_END()
+	}
+};
 
-		/** HMAC_MD5 Authentication */
+static struct unit_test_suite cryptodev_hmac_md5_auth_testsuite  = {
+	.suite_name = "HMAC_MD5 Authentication Test Suite",
+	.setup = hmac_md5_auth_testsuite_setup,
+	.unit_test_cases = {
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_MD5_HMAC_generate_case_1),
 		TEST_CASE_ST(ut_setup, ut_teardown,
@@ -13701,7 +14279,14 @@ static struct unit_test_suite cryptodev_testsuite  = {
 			test_MD5_HMAC_generate_case_2),
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_MD5_HMAC_verify_case_2),
+		TEST_CASES_END()
+	}
+};
 
+static struct unit_test_suite cryptodev_kasumi_testsuite  = {
+	.suite_name = "Kasumi Test Suite",
+	.setup = kasumi_testsuite_setup,
+	.unit_test_cases = {
 		/** KASUMI hash only (UIA1) */
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_kasumi_hash_generate_test_case_1),
@@ -13786,17 +14371,26 @@ static struct unit_test_suite cryptodev_testsuite  = {
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_kasumi_auth_cipher_verify_test_case_2_oop_sgl),
 
-		/** ESN Testcase */
+		TEST_CASES_END()
+	}
+};
+
+static struct unit_test_suite cryptodev_esn_testsuite  = {
+	.suite_name = "ESN Test Suite",
+	.setup = esn_testsuite_setup,
+	.unit_test_cases = {
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			auth_encrypt_AES128CBC_HMAC_SHA1_esn_check),
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			auth_decrypt_AES128CBC_HMAC_SHA1_esn_check),
+		TEST_CASES_END()
+	}
+};
 
-		/** Negative tests */
-		TEST_CASE_ST(ut_setup, ut_teardown,
-			authentication_verify_HMAC_SHA1_fail_data_corrupt),
-		TEST_CASE_ST(ut_setup, ut_teardown,
-			authentication_verify_HMAC_SHA1_fail_tag_corrupt),
+static struct unit_test_suite cryptodev_negative_aes_gcm_testsuite  = {
+	.suite_name = "Negative AES GCM Test Suite",
+	.setup = negative_aes_gcm_testsuite_setup,
+	.unit_test_cases = {
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_AES_GCM_auth_encryption_fail_iv_corrupt),
 		TEST_CASE_ST(ut_setup, ut_teardown,
@@ -13821,16 +14415,28 @@ static struct unit_test_suite cryptodev_testsuite  = {
 			test_AES_GCM_auth_decryption_fail_aad_corrupt),
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_AES_GCM_auth_decryption_fail_tag_corrupt),
+
+		TEST_CASES_END()
+	}
+};
+
+static struct unit_test_suite cryptodev_negative_aes_gmac_testsuite  = {
+	.suite_name = "Negative AES GMAC Test Suite",
+	.setup = negative_aes_gmac_testsuite_setup,
+	.unit_test_cases = {
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			authentication_verify_AES128_GMAC_fail_data_corrupt),
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			authentication_verify_AES128_GMAC_fail_tag_corrupt),
-		TEST_CASE_ST(ut_setup, ut_teardown,
-			auth_decryption_AES128CBC_HMAC_SHA1_fail_data_corrupt),
-		TEST_CASE_ST(ut_setup, ut_teardown,
-			auth_decryption_AES128CBC_HMAC_SHA1_fail_tag_corrupt),
 
-		/** Mixed CIPHER + HASH algorithms */
+		TEST_CASES_END()
+	}
+};
+
+static struct unit_test_suite cryptodev_mixed_cipher_hash_testsuite  = {
+	.suite_name = "Mixed CIPHER + HASH algorithms Test Suite",
+	.setup = mixed_cipher_hash_testsuite_setup,
+	.unit_test_cases = {
 		/** AUTH AES CMAC + CIPHER AES CTR */
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_aes_cmac_aes_ctr_digest_enc_test_case_1),
@@ -13843,11 +14449,11 @@ static struct unit_test_suite cryptodev_testsuite  = {
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_verify_aes_cmac_aes_ctr_digest_enc_test_case_1),
 		TEST_CASE_ST(ut_setup, ut_teardown,
-		       test_verify_aes_cmac_aes_ctr_digest_enc_test_case_1_oop),
+			test_verify_aes_cmac_aes_ctr_digest_enc_test_case_1_oop),
 		TEST_CASE_ST(ut_setup, ut_teardown,
-		       test_verify_aes_cmac_aes_ctr_digest_enc_test_case_1_sgl),
+			test_verify_aes_cmac_aes_ctr_digest_enc_test_case_1_sgl),
 		TEST_CASE_ST(ut_setup, ut_teardown,
-		   test_verify_aes_cmac_aes_ctr_digest_enc_test_case_1_oop_sgl),
+			test_verify_aes_cmac_aes_ctr_digest_enc_test_case_1_oop_sgl),
 
 		/** AUTH ZUC + CIPHER SNOW3G */
 		TEST_CASE_ST(ut_setup, ut_teardown,
@@ -13910,165 +14516,86 @@ static struct unit_test_suite cryptodev_testsuite  = {
 			test_auth_aes_cmac_cipher_null_test_case_1),
 		TEST_CASE_ST(ut_setup, ut_teardown,
 			test_verify_auth_aes_cmac_cipher_null_test_case_1),
-
-#ifdef RTE_LIB_SECURITY
-		TEST_CASE_ST(ut_setup_security, ut_teardown,
-			test_PDCP_PROTO_all),
-		TEST_CASE_ST(ut_setup_security, ut_teardown,
-			test_DOCSIS_PROTO_all),
-#endif
-		TEST_CASE_ST(ut_setup, ut_teardown, test_enq_callback_setup),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_deq_callback_setup),
-		TEST_CASES_END() /**< NULL terminate unit test array */
-	}
-};
-
-static struct unit_test_suite cryptodev_virtio_testsuite = {
-	.suite_name = "Crypto VIRTIO Unit Test Suite",
-	.setup = testsuite_setup,
-	.teardown = testsuite_teardown,
-	.unit_test_cases = {
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_cipheronly_all),
-
-		TEST_CASES_END() /**< NULL terminate unit test array */
-	}
-};
-
-static struct unit_test_suite cryptodev_caam_jr_testsuite  = {
-	.suite_name = "Crypto CAAM JR Unit Test Suite",
-	.setup = testsuite_setup,
-	.teardown = testsuite_teardown,
-	.unit_test_cases = {
-		TEST_CASE_ST(ut_setup, ut_teardown,
-			     test_device_configure_invalid_dev_id),
-		TEST_CASE_ST(ut_setup, ut_teardown,
-			     test_multi_session),
-
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_3DES_chain_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_cipheronly_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_3DES_cipheronly_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_authonly_all),
-
-		TEST_CASES_END() /**< NULL terminate unit test array */
+		TEST_CASES_END()
 	}
 };
 
-static struct unit_test_suite cryptodev_mrvl_testsuite  = {
-	.suite_name = "Crypto Device Marvell Component Test Suite",
-	.setup = testsuite_setup,
-	.teardown = testsuite_teardown,
-	.unit_test_cases = {
-		TEST_CASE_ST(ut_setup, ut_teardown, test_multi_session),
-		TEST_CASE_ST(ut_setup, ut_teardown,
-				test_multi_session_random_usage),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_cipheronly_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_authonly_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_3DES_chain_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_3DES_cipheronly_all),
+static int
+run_cryptodev_testsuite(const char *pmd_name)
+{
+	uint8_t ret, j, i = 0;
+	struct unit_test_suite *static_suites[] = {
+		&cryptodev_multi_session_testsuite,
+		&cryptodev_null_testsuite,
+		&cryptodev_aes_ccm_auth_testsuite,
+		&cryptodev_aes_gcm_auth_testsuite,
+		&cryptodev_aes_gmac_auth_testsuite,
+		&cryptodev_snow3g_testsuite,
+		&cryptodev_chacha20_poly1305_testsuite,
+		&cryptodev_zuc_testsuite,
+		&cryptodev_hmac_md5_auth_testsuite,
+		&cryptodev_kasumi_testsuite,
+		&cryptodev_esn_testsuite,
+		&cryptodev_negative_aes_gcm_testsuite,
+		&cryptodev_negative_aes_gmac_testsuite,
+		&cryptodev_mixed_cipher_hash_testsuite,
+		&cryptodev_negative_hmac_sha1_testsuite,
+		&cryptodev_gen_testsuite,
+#ifdef RTE_LIB_SECURITY
+		&pdcp_proto_testsuite,
+		&docsis_proto_testsuite,
+#endif
+		&end_testsuite
+	};
+	static struct unit_test_suite ts = {
+		.suite_name = "Cryptodev Unit Test Suite",
+		.setup = testsuite_setup,
+		.teardown = testsuite_teardown,
+		.unit_test_cases = {TEST_CASES_END()}
+	};
 
-		/** Negative tests */
-		TEST_CASE_ST(ut_setup, ut_teardown,
-			authentication_verify_HMAC_SHA1_fail_data_corrupt),
-		TEST_CASE_ST(ut_setup, ut_teardown,
-			authentication_verify_HMAC_SHA1_fail_tag_corrupt),
-		TEST_CASE_ST(ut_setup, ut_teardown,
-			auth_decryption_AES128CBC_HMAC_SHA1_fail_data_corrupt),
-		TEST_CASE_ST(ut_setup, ut_teardown,
-			auth_decryption_AES128CBC_HMAC_SHA1_fail_tag_corrupt),
+	gbl_driver_id = rte_cryptodev_driver_id_get(pmd_name);
 
-		TEST_CASES_END() /**< NULL terminate unit test array */
+	if (gbl_driver_id == -1) {
+		RTE_LOG(ERR, USER1, "%s PMD must be loaded.\n", pmd_name);
+		return TEST_FAILED;
 	}
-};
 
-static struct unit_test_suite cryptodev_ccp_testsuite  = {
-	.suite_name = "Crypto Device CCP Unit Test Suite",
-	.setup = testsuite_setup,
-	.teardown = testsuite_teardown,
-	.unit_test_cases = {
-		TEST_CASE_ST(ut_setup, ut_teardown, test_multi_session),
-		TEST_CASE_ST(ut_setup, ut_teardown,
-				test_multi_session_random_usage),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_cipheronly_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_3DES_chain_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_3DES_cipheronly_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_authonly_all),
+	ts.unit_test_suites = malloc(sizeof(struct unit_test_suite *) *
+			RTE_DIM(static_suites));
 
-		/** Negative tests */
-		TEST_CASE_ST(ut_setup, ut_teardown,
-			authentication_verify_HMAC_SHA1_fail_data_corrupt),
-		TEST_CASE_ST(ut_setup, ut_teardown,
-			authentication_verify_HMAC_SHA1_fail_tag_corrupt),
-		TEST_CASE_ST(ut_setup, ut_teardown,
-			auth_decryption_AES128CBC_HMAC_SHA1_fail_data_corrupt),
-		TEST_CASE_ST(ut_setup, ut_teardown,
-			auth_decryption_AES128CBC_HMAC_SHA1_fail_tag_corrupt),
+	ADD_STATIC_TESTSUITE(i, ts, static_suites, RTE_DIM(static_suites));
+	ret = unit_test_suite_runner(&ts);
 
-		TEST_CASES_END() /**< NULL terminate unit test array */
-	}
-};
+	free(ts.unit_test_suites);
+	return ret;
+}
 
 static int
 test_cryptodev_qat(void /*argv __rte_unused, int argc __rte_unused*/)
 {
-	gbl_driver_id =	rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "QAT PMD must be loaded.\n");
-		return TEST_SKIPPED;
-	}
-
-	return unit_test_suite_runner(&cryptodev_testsuite);
+	return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD));
 }
 
 static int
 test_cryptodev_virtio(void /*argv __rte_unused, int argc __rte_unused*/)
 {
-	gbl_driver_id =	rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_VIRTIO_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "VIRTIO PMD must be loaded.\n");
-		return TEST_FAILED;
-	}
-
-	return unit_test_suite_runner(&cryptodev_virtio_testsuite);
+	return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_VIRTIO_PMD));
 }
 
 static int
 test_cryptodev_aesni_mb(void /*argv __rte_unused, int argc __rte_unused*/)
 {
-	gbl_driver_id =	rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "AESNI MB PMD must be loaded.\n");
-		return TEST_SKIPPED;
-	}
-
-	return unit_test_suite_runner(&cryptodev_testsuite);
+	return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
 }
 
 static int
 test_cryptodev_cpu_aesni_mb(void)
 {
 	int32_t rc;
-	enum rte_security_session_action_type at;
-
-	gbl_driver_id =	rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "AESNI MB PMD must be loaded.\n");
-		return TEST_SKIPPED;
-	}
-
-	at = gbl_action_type;
+	enum rte_security_session_action_type at = gbl_action_type;
 	gbl_action_type = RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO;
-	rc = unit_test_suite_runner(&cryptodev_testsuite);
+	rc = run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
 	gbl_action_type = at;
 	return rc;
 }
@@ -14076,48 +14603,22 @@ test_cryptodev_cpu_aesni_mb(void)
 static int
 test_cryptodev_openssl(void)
 {
-	gbl_driver_id = rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_OPENSSL_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "OPENSSL PMD must be loaded.\n");
-		return TEST_SKIPPED;
-	}
-
-	return unit_test_suite_runner(&cryptodev_testsuite);
+	return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_OPENSSL_PMD));
 }
 
 static int
 test_cryptodev_aesni_gcm(void)
 {
-	gbl_driver_id = rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "AESNI GCM PMD must be loaded.\n");
-		return TEST_SKIPPED;
-	}
-
-	return unit_test_suite_runner(&cryptodev_testsuite);
+	return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD));
 }
 
 static int
 test_cryptodev_cpu_aesni_gcm(void)
 {
 	int32_t rc;
-	enum rte_security_session_action_type at;
-
-	gbl_driver_id = rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "AESNI GCM PMD must be loaded.\n");
-		return TEST_SKIPPED;
-	}
-
-	at = gbl_action_type;
+	enum rte_security_session_action_type at = gbl_action_type;
 	gbl_action_type = RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO;
-	rc = unit_test_suite_runner(&cryptodev_testsuite);
+	rc  = run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD));
 	gbl_action_type = at;
 	return rc;
 }
@@ -14125,85 +14626,37 @@ test_cryptodev_cpu_aesni_gcm(void)
 static int
 test_cryptodev_null(void)
 {
-	gbl_driver_id = rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_NULL_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "NULL PMD must be loaded.\n");
-		return TEST_SKIPPED;
-	}
-
-	return unit_test_suite_runner(&cryptodev_testsuite);
+	return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_NULL_PMD));
 }
 
 static int
 test_cryptodev_sw_snow3g(void /*argv __rte_unused, int argc __rte_unused*/)
 {
-	gbl_driver_id = rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "SNOW3G PMD must be loaded.\n");
-		return TEST_SKIPPED;
-	}
-
-	return unit_test_suite_runner(&cryptodev_testsuite);
+	return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_SNOW3G_PMD));
 }
 
 static int
 test_cryptodev_sw_kasumi(void /*argv __rte_unused, int argc __rte_unused*/)
 {
-	gbl_driver_id = rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_KASUMI_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "ZUC PMD must be loaded.\n");
-		return TEST_SKIPPED;
-	}
-
-	return unit_test_suite_runner(&cryptodev_testsuite);
+	return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_KASUMI_PMD));
 }
 
 static int
 test_cryptodev_sw_zuc(void /*argv __rte_unused, int argc __rte_unused*/)
 {
-	gbl_driver_id = rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_ZUC_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "ZUC PMD must be loaded.\n");
-		return TEST_SKIPPED;
-	}
-
-	return unit_test_suite_runner(&cryptodev_testsuite);
+	return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_ZUC_PMD));
 }
 
 static int
 test_cryptodev_armv8(void)
 {
-	gbl_driver_id = rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_ARMV8_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "ARMV8 PMD must be loaded.\n");
-		return TEST_SKIPPED;
-	}
-
-	return unit_test_suite_runner(&cryptodev_testsuite);
+	return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_ARMV8_PMD));
 }
 
 static int
 test_cryptodev_mrvl(void)
 {
-	gbl_driver_id = rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_MVSAM_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "MVSAM PMD must be loaded.\n");
-		return TEST_SKIPPED;
-	}
-
-	return unit_test_suite_runner(&cryptodev_mrvl_testsuite);
+	return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_MVSAM_PMD));
 }
 
 #ifdef RTE_CRYPTO_SCHEDULER
@@ -14211,6 +14664,85 @@ test_cryptodev_mrvl(void)
 static int
 test_cryptodev_scheduler(void /*argv __rte_unused, int argc __rte_unused*/)
 {
+	uint8_t ret, j, i = 0;
+	static struct unit_test_suite scheduler_multicore = {
+		.suite_name = "Scheduler Multicore Unit Test Suite",
+		.setup = scheduler_multicore_testsuite_setup,
+		.teardown = scheduler_mode_testsuite_teardown,
+		.unit_test_cases = {
+			TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
+			TEST_CASE_ST(ut_setup, ut_teardown,
+					test_AES_cipheronly_all),
+			TEST_CASE_ST(ut_setup, ut_teardown, test_authonly_all),
+			TEST_CASES_END()
+		}
+	};
+	static struct unit_test_suite scheduler_round_robin = {
+		.suite_name = "Scheduler Round Robin Unit Test Suite",
+		.setup = scheduler_roundrobin_testsuite_setup,
+		.teardown = scheduler_mode_testsuite_teardown,
+		.unit_test_cases = {
+			TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
+			TEST_CASE_ST(ut_setup, ut_teardown,
+					test_AES_cipheronly_all),
+			TEST_CASE_ST(ut_setup, ut_teardown, test_authonly_all),
+			TEST_CASES_END()
+		}
+	};
+	static struct unit_test_suite scheduler_failover = {
+		.suite_name = "Scheduler Failover Unit Test Suite",
+		.setup = scheduler_failover_testsuite_setup,
+		.teardown = scheduler_mode_testsuite_teardown,
+		.unit_test_cases = {
+			TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
+			TEST_CASE_ST(ut_setup, ut_teardown,
+					test_AES_cipheronly_all),
+			TEST_CASE_ST(ut_setup, ut_teardown, test_authonly_all),
+			TEST_CASES_END()
+		}
+	};
+	static struct unit_test_suite scheduler_pkt_size_distr = {
+		.suite_name = "Scheduler Pkt Size Distr Unit Test Suite",
+		.setup = scheduler_pkt_size_distr_testsuite_setup,
+		.teardown = scheduler_mode_testsuite_teardown,
+		.unit_test_cases = {
+			TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
+			TEST_CASE_ST(ut_setup, ut_teardown,
+					test_AES_cipheronly_all),
+			TEST_CASE_ST(ut_setup, ut_teardown, test_authonly_all),
+			TEST_CASES_END()
+		}
+	};
+	struct unit_test_suite *sched_mode_suites[] = {
+		&scheduler_multicore,
+		&scheduler_round_robin,
+		&scheduler_failover,
+		&scheduler_pkt_size_distr
+	};
+	static struct unit_test_suite scheduler_config = {
+		.suite_name = "Crypto Device Scheduler Config Unit Test Suite",
+		.unit_test_cases = {
+			TEST_CASE(test_scheduler_attach_slave_op),
+			TEST_CASE(test_scheduler_mode_multicore_op),
+			TEST_CASE(test_scheduler_mode_roundrobin_op),
+			TEST_CASE(test_scheduler_mode_failover_op),
+			TEST_CASE(test_scheduler_mode_pkt_size_distr_op),
+			TEST_CASE(test_scheduler_detach_slave_op),
+
+			TEST_CASES_END() /**< NULL terminate array */
+		}
+	};
+	struct unit_test_suite *static_suites[] = {
+		&scheduler_config,
+		&end_testsuite
+	};
+	static struct unit_test_suite ts = {
+		.suite_name = "Scheduler Unit Test Suite",
+		.setup = scheduler_testsuite_setup,
+		.teardown = testsuite_teardown,
+		.unit_test_cases = {TEST_CASES_END()}
+	};
+
 	gbl_driver_id =	rte_cryptodev_driver_id_get(
 			RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD));
 
@@ -14223,8 +14755,17 @@ test_cryptodev_scheduler(void /*argv __rte_unused, int argc __rte_unused*/)
 				RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD)) == -1) {
 		RTE_LOG(ERR, USER1, "AESNI MB PMD must be loaded.\n");
 		return TEST_SKIPPED;
-}
-	return unit_test_suite_runner(&cryptodev_scheduler_testsuite);
+	}
+
+	ts.unit_test_suites = malloc(sizeof(struct unit_test_suite *) *
+			(RTE_DIM(static_suites) + RTE_DIM(sched_mode_suites)));
+	ADD_STATIC_TESTSUITE(i, ts, sched_mode_suites,
+			RTE_DIM(sched_mode_suites));
+	ADD_STATIC_TESTSUITE(i, ts, static_suites, RTE_DIM(static_suites));
+	ret = unit_test_suite_runner(&ts);
+
+	free(ts.unit_test_suites);
+	return ret;
 }
 
 REGISTER_TEST_COMMAND(cryptodev_scheduler_autotest, test_cryptodev_scheduler);
@@ -14234,109 +14775,49 @@ REGISTER_TEST_COMMAND(cryptodev_scheduler_autotest, test_cryptodev_scheduler);
 static int
 test_cryptodev_dpaa2_sec(void /*argv __rte_unused, int argc __rte_unused*/)
 {
-	gbl_driver_id =	rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_DPAA2_SEC_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "DPAA2 SEC PMD must be loaded.\n");
-		return TEST_SKIPPED;
-	}
-
-	return unit_test_suite_runner(&cryptodev_testsuite);
+	return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_DPAA2_SEC_PMD));
 }
 
 static int
 test_cryptodev_dpaa_sec(void /*argv __rte_unused, int argc __rte_unused*/)
 {
-	gbl_driver_id =	rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_DPAA_SEC_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "DPAA SEC PMD must be loaded.\n");
-		return TEST_SKIPPED;
-	}
-
-	return unit_test_suite_runner(&cryptodev_testsuite);
+	return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_DPAA_SEC_PMD));
 }
 
 static int
 test_cryptodev_ccp(void)
 {
-	gbl_driver_id = rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_CCP_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "CCP PMD must be loaded.\n");
-		return TEST_FAILED;
-	}
-
-	return unit_test_suite_runner(&cryptodev_ccp_testsuite);
+	return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_CCP_PMD));
 }
 
 static int
 test_cryptodev_octeontx(void)
 {
-	gbl_driver_id =	rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_OCTEONTX_SYM_PMD));
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "OCTEONTX PMD must be loaded.\n");
-		return TEST_FAILED;
-	}
-	return unit_test_suite_runner(&cryptodev_testsuite);
+	return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_OCTEONTX_SYM_PMD));
 }
 
 static int
 test_cryptodev_octeontx2(void)
 {
-	gbl_driver_id =	rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_OCTEONTX2_PMD));
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "OCTEON TX2 PMD must be loaded.\n");
-		return TEST_FAILED;
-	}
-	return unit_test_suite_runner(&cryptodev_testsuite);
+	return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_OCTEONTX2_PMD));
 }
 
 static int
 test_cryptodev_caam_jr(void /*argv __rte_unused, int argc __rte_unused*/)
 {
-	gbl_driver_id =	rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_CAAM_JR_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "CAAM_JR PMD must be loaded.\n");
-		return TEST_FAILED;
-	}
-
-	return unit_test_suite_runner(&cryptodev_caam_jr_testsuite);
+	return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_CAAM_JR_PMD));
 }
 
 static int
 test_cryptodev_nitrox(void)
 {
-	gbl_driver_id =	rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_NITROX_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "NITROX PMD must be loaded.\n");
-		return TEST_FAILED;
-	}
-
-	return unit_test_suite_runner(&cryptodev_testsuite);
+	return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_NITROX_PMD));
 }
 
 static int
 test_cryptodev_bcmfs(void)
 {
-	gbl_driver_id =	rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_BCMFS_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "BCMFS PMD must be loaded.\n");
-		return TEST_FAILED;
-	}
-
-	return unit_test_suite_runner(&cryptodev_testsuite);
+	return run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_BCMFS_PMD));
 }
 
 static int
@@ -14344,16 +14825,8 @@ test_cryptodev_qat_raw_api(void /*argv __rte_unused, int argc __rte_unused*/)
 {
 	int ret;
 
-	gbl_driver_id =	rte_cryptodev_driver_id_get(
-			RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD));
-
-	if (gbl_driver_id == -1) {
-		RTE_LOG(ERR, USER1, "QAT PMD must be loaded.\n");
-		return TEST_SKIPPED;
-	}
-
 	global_api_test_type = CRYPTODEV_RAW_API_TEST;
-	ret = unit_test_suite_runner(&cryptodev_testsuite);
+	ret = run_cryptodev_testsuite(RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD));
 	global_api_test_type = CRYPTODEV_API_TEST;
 
 	return ret;
diff --git a/app/test/test_cryptodev.h b/app/test/test_cryptodev.h
index bf4f6c8d62..5c41e36f44 100644
--- a/app/test/test_cryptodev.h
+++ b/app/test/test_cryptodev.h
@@ -222,4 +222,16 @@ process_sym_raw_dp_op(uint8_t dev_id, uint16_t qp_id,
 		struct rte_crypto_op *op, uint8_t is_cipher, uint8_t is_auth,
 		uint8_t len_in_bits, uint8_t cipher_iv_len);
 
+int
+check_cipher_capabilities_supported(const enum rte_crypto_cipher_algorithm *ciphers,
+		uint16_t num_ciphers);
+
+int
+check_auth_capabilities_supported(const enum rte_crypto_auth_algorithm *auths,
+		uint16_t num_auths);
+
+int
+check_aead_capabilities_supported(const enum rte_crypto_aead_algorithm *aeads,
+		uint16_t num_aeads);
+
 #endif /* TEST_CRYPTODEV_H_ */
-- 
2.25.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH v3 3/7] test/crypto: refactor to use sub-testsuites
  2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 3/7] test/crypto: refactor to use sub-testsuites Ciara Power
@ 2021-04-30 20:59   ` Doherty, Declan
  0 siblings, 0 replies; 15+ messages in thread
From: Doherty, Declan @ 2021-04-30 20:59 UTC (permalink / raw)
  To: Ciara Power, dev
  Cc: thomas, gakhil, aconole, hemant.agrawal, anoobj, ruifeng.wang,
	asomalap, ajit.khaparde, g.singh



On 23/04/2021 5:18 PM, Ciara Power wrote:
> The existing implementation runs a giant cryptodev testsuite for most
> autotests, which in turns runs one setup function regardless of device.
> 
> This is now broken down into multiple testsuites,
> that are used as sub-testsuites. Each autotest runs a general crypto
> parent test suite, to which the sub-testsuites are added.
> 
> For example, the AESNI_MB test runs "Cryptodev Unit Test Suite",
> which has a setup function only to configure testsuite params.
> Creation of vdevs in the setup function is no longer supported,
> it is expected the user does this when running the app.
> This autotest previously just ran the cryptodev_testsuite,
> but now has the smaller sub-testsuites added to the parent suite instead.
> The same test cases are being run as before.
> 
> The scheduler autotest creates its own parent testsuite with nested
> sub-testsuites, rather than using the cryptodev testsuite mentioned above.
> This is due to it being more complex in execution,
> by requiring setting different modes before running tests.
> The scheduler autotest no longer requires the extra test cases to
> attach/set mode/detach when running the blockcipher test cases for
> each mode. The attach/set mode/detach functionality is now tested in a
> sub-testsuite. When running the sub-testsuites for each mode,
> the attach/set mode/detach happens in the setup and teardown functions
> for that sub-testsuite.
> 
> Signed-off-by: Ciara Power <ciara.power@intel.com>
> 
....
> 

Hey Ciara, can you make sure to detail the new requirement in the unit 
test documentation about the need to create the sw crypto pmds using the 
--vdev EAL arguments.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [dpdk-dev] [PATCH v3 4/7] test/crypto: replace unsupported with skipped
  2021-04-23 16:18 [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework Ciara Power
                   ` (2 preceding siblings ...)
  2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 3/7] test/crypto: refactor to use sub-testsuites Ciara Power
@ 2021-04-23 16:18 ` Ciara Power
  2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 5/7] test/crypto: move testsuite params to header file Ciara Power
                   ` (5 subsequent siblings)
  9 siblings, 0 replies; 15+ messages in thread
From: Ciara Power @ 2021-04-23 16:18 UTC (permalink / raw)
  To: dev
  Cc: thomas, declan.doherty, gakhil, aconole, hemant.agrawal, anoobj,
	ruifeng.wang, asomalap, ajit.khaparde, g.singh, Ciara Power

Testcases were previously using -ENOTSUP and TEST_SKIPPED return
statuses interchangeably. Both resulted in the testcase not being run.

These return statuses are now standardised to TEST_SKIPPED.

Signed-off-by: Ciara Power <ciara.power@intel.com>
---
 app/test/test_cryptodev.c      | 464 ++++++++++++++++-----------------
 app/test/test_cryptodev_asym.c |  18 +-
 2 files changed, 241 insertions(+), 241 deletions(-)

diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index 4fe30ddb21..2c9477680e 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -2060,12 +2060,12 @@ test_AES_CBC_HMAC_SHA1_encrypt_digest(void)
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_SHA1_HMAC;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_AES_CBC;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Generate test mbuf data and space for digest */
 	ut_params->ibuf = setup_test_string(ts_params->mbuf_pool,
@@ -2593,7 +2593,7 @@ create_wireless_algo_cipher_auth_session(uint8_t dev_id,
 			&ut_params->cipher_xform,
 			ts_params->session_priv_mpool);
 	if (status == -ENOTSUP)
-		return status;
+		return TEST_SKIPPED;
 
 	TEST_ASSERT_EQUAL(status, 0, "session init failed");
 	return 0;
@@ -2656,7 +2656,7 @@ create_wireless_cipher_auth_session(uint8_t dev_id,
 			&ut_params->cipher_xform,
 			ts_params->session_priv_mpool);
 	if (status == -ENOTSUP)
-		return status;
+		return TEST_SKIPPED;
 
 	TEST_ASSERT_EQUAL(status, 0, "session init failed");
 	TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed");
@@ -2732,7 +2732,7 @@ create_wireless_algo_auth_cipher_session(uint8_t dev_id,
 				ts_params->session_priv_mpool);
 
 	if (status == -ENOTSUP)
-		return status;
+		return TEST_SKIPPED;
 
 	TEST_ASSERT_EQUAL(status, 0, "session init failed");
 
@@ -3076,17 +3076,17 @@ test_snow3g_authentication(const struct snow3g_hash_test_data *tdata)
 	if (!(feat_flags & RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA) &&
 			((tdata->validAuthLenInBits.len % 8) != 0)) {
 		printf("Device doesn't support NON-Byte Aligned Data.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3094,7 +3094,7 @@ test_snow3g_authentication(const struct snow3g_hash_test_data *tdata)
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_SNOW3G_UIA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_hash_session(ts_params->valid_devs[0],
@@ -3167,17 +3167,17 @@ test_snow3g_authentication_verify(const struct snow3g_hash_test_data *tdata)
 	if (!(feat_flags & RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA) &&
 			((tdata->validAuthLenInBits.len % 8) != 0)) {
 		printf("Device doesn't support NON-Byte Aligned Data.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3185,7 +3185,7 @@ test_snow3g_authentication_verify(const struct snow3g_hash_test_data *tdata)
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_SNOW3G_UIA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_hash_session(ts_params->valid_devs[0],
@@ -3258,11 +3258,11 @@ test_kasumi_authentication(const struct kasumi_hash_test_data *tdata)
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3270,7 +3270,7 @@ test_kasumi_authentication(const struct kasumi_hash_test_data *tdata)
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_KASUMI_F9;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create KASUMI session */
 	retval = create_wireless_algo_hash_session(ts_params->valid_devs[0],
@@ -3347,11 +3347,11 @@ test_kasumi_authentication_verify(const struct kasumi_hash_test_data *tdata)
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3359,7 +3359,7 @@ test_kasumi_authentication_verify(const struct kasumi_hash_test_data *tdata)
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_KASUMI_F9;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create KASUMI session */
 	retval = create_wireless_algo_hash_session(ts_params->valid_devs[0],
@@ -3571,11 +3571,11 @@ test_kasumi_encryption(const struct kasumi_test_data *tdata)
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -3583,7 +3583,7 @@ test_kasumi_encryption(const struct kasumi_test_data *tdata)
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
@@ -3667,7 +3667,7 @@ test_kasumi_encryption_sgl(const struct kasumi_test_data *tdata)
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -3676,17 +3676,17 @@ test_kasumi_encryption_sgl(const struct kasumi_test_data *tdata)
 	if (!(feat_flags & RTE_CRYPTODEV_FF_IN_PLACE_SGL)) {
 		printf("Device doesn't support in-place scatter-gather. "
 				"Test Skipped.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
@@ -3767,13 +3767,13 @@ test_kasumi_encryption_oop(const struct kasumi_test_data *tdata)
 	/* Data-path service does not support OOP */
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
@@ -3854,13 +3854,13 @@ test_kasumi_encryption_oop_sgl(const struct kasumi_test_data *tdata)
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -3869,7 +3869,7 @@ test_kasumi_encryption_oop_sgl(const struct kasumi_test_data *tdata)
 		printf("Device doesn't support out-of-place scatter-gather "
 				"in both input and output mbufs. "
 				"Test Skipped.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	/* Create KASUMI session */
@@ -3945,13 +3945,13 @@ test_kasumi_decryption_oop(const struct kasumi_test_data *tdata)
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
@@ -4029,11 +4029,11 @@ test_kasumi_decryption(const struct kasumi_test_data *tdata)
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -4041,7 +4041,7 @@ test_kasumi_decryption(const struct kasumi_test_data *tdata)
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
@@ -4121,11 +4121,11 @@ test_snow3g_encryption(const struct snow3g_test_data *tdata)
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -4133,7 +4133,7 @@ test_snow3g_encryption(const struct snow3g_test_data *tdata)
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
@@ -4211,13 +4211,13 @@ test_snow3g_encryption_oop(const struct snow3g_test_data *tdata)
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
@@ -4300,13 +4300,13 @@ test_snow3g_encryption_oop_sgl(const struct snow3g_test_data *tdata)
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -4316,7 +4316,7 @@ test_snow3g_encryption_oop_sgl(const struct snow3g_test_data *tdata)
 		printf("Device doesn't support out-of-place scatter-gather "
 				"in both input and output mbufs. "
 				"Test Skipped.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	/* Create SNOW 3G session */
@@ -4416,7 +4416,7 @@ test_snow3g_encryption_offset_oop(const struct snow3g_test_data *tdata)
 	if (!(feat_flags & RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA) &&
 			((tdata->validDataLenInBits.len % 8) != 0)) {
 		printf("Device doesn't support NON-Byte Aligned Data.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	/* Verify the capabilities */
@@ -4425,13 +4425,13 @@ test_snow3g_encryption_offset_oop(const struct snow3g_test_data *tdata)
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
@@ -4531,11 +4531,11 @@ static int test_snow3g_decryption(const struct snow3g_test_data *tdata)
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -4543,7 +4543,7 @@ static int test_snow3g_decryption(const struct snow3g_test_data *tdata)
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
@@ -4618,13 +4618,13 @@ static int test_snow3g_decryption_oop(const struct snow3g_test_data *tdata)
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
@@ -4710,17 +4710,17 @@ test_zuc_cipher_auth(const struct wireless_test_data *tdata)
 			((tdata->validAuthLenInBits.len % 8 != 0) ||
 			(tdata->validDataLenInBits.len % 8 != 0))) {
 		printf("Device doesn't support NON-Byte Aligned Data.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Check if device supports ZUC EEA3 */
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
@@ -4728,7 +4728,7 @@ test_zuc_cipher_auth(const struct wireless_test_data *tdata)
 
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Check if device supports ZUC EIA3 */
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
@@ -4736,7 +4736,7 @@ test_zuc_cipher_auth(const struct wireless_test_data *tdata)
 
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create ZUC session */
 	retval = create_zuc_cipher_auth_encrypt_generate_session(
@@ -4817,11 +4817,11 @@ test_snow3g_cipher_auth(const struct snow3g_test_data *tdata)
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -4829,12 +4829,12 @@ test_snow3g_cipher_auth(const struct snow3g_test_data *tdata)
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_SNOW3G_UIA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create SNOW 3G session */
 	retval = create_wireless_algo_cipher_auth_session(ts_params->valid_devs[0],
@@ -4933,15 +4933,15 @@ test_snow3g_auth_cipher(const struct snow3g_test_data *tdata,
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_SNOW3G_UIA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -4950,16 +4950,16 @@ test_snow3g_auth_cipher(const struct snow3g_test_data *tdata,
 	if (op_mode == OUT_OF_PLACE) {
 		if (!(feat_flags & RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED)) {
 			printf("Device doesn't support digest encrypted.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 	}
 
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	/* Create SNOW 3G session */
@@ -5125,15 +5125,15 @@ test_snow3g_auth_cipher_sgl(const struct snow3g_test_data *tdata,
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_SNOW3G_UIA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_SNOW3G_UEA2;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -5143,24 +5143,24 @@ test_snow3g_auth_cipher_sgl(const struct snow3g_test_data *tdata,
 		if (!(feat_flags & RTE_CRYPTODEV_FF_IN_PLACE_SGL)) {
 			printf("Device doesn't support in-place scatter-gather "
 					"in both input and output mbufs.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 			printf("Device doesn't support RAW data-path APIs.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 	} else {
 		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 		if (!(feat_flags & RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED)) {
 			printf("Device doesn't support digest encrypted.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 	}
 
@@ -5332,12 +5332,12 @@ test_kasumi_auth_cipher(const struct kasumi_test_data *tdata,
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_KASUMI_F9;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -5346,18 +5346,18 @@ test_kasumi_auth_cipher(const struct kasumi_test_data *tdata,
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (op_mode == OUT_OF_PLACE) {
 		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED)) {
 			printf("Device doesn't support digest encrypted.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 	}
 
@@ -5526,15 +5526,15 @@ test_kasumi_auth_cipher_sgl(const struct kasumi_test_data *tdata,
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_KASUMI_F9;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -5544,24 +5544,24 @@ test_kasumi_auth_cipher_sgl(const struct kasumi_test_data *tdata,
 		if (!(feat_flags & RTE_CRYPTODEV_FF_IN_PLACE_SGL)) {
 			printf("Device doesn't support in-place scatter-gather "
 					"in both input and output mbufs.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 			printf("Device doesn't support RAW data-path APIs.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 	} else {
 		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 		if (!(feat_flags & RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED)) {
 			printf("Device doesn't support digest encrypted.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 	}
 
@@ -5729,11 +5729,11 @@ test_kasumi_cipher_auth(const struct kasumi_test_data *tdata)
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -5741,12 +5741,12 @@ test_kasumi_cipher_auth(const struct kasumi_test_data *tdata)
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_KASUMI_F9;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_KASUMI_F8;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create KASUMI session */
 	retval = create_wireless_algo_cipher_auth_session(
@@ -5845,11 +5845,11 @@ test_zuc_encryption(const struct wireless_test_data *tdata)
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	struct rte_cryptodev_sym_capability_idx cap_idx;
 
@@ -5859,7 +5859,7 @@ test_zuc_encryption(const struct wireless_test_data *tdata)
 
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create ZUC session */
 	retval = create_wireless_algo_cipher_session(ts_params->valid_devs[0],
@@ -5941,10 +5941,10 @@ test_zuc_encryption_sgl(const struct wireless_test_data *tdata)
 
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -5953,13 +5953,13 @@ test_zuc_encryption_sgl(const struct wireless_test_data *tdata)
 	if (!(feat_flags & RTE_CRYPTODEV_FF_IN_PLACE_SGL)) {
 		printf("Device doesn't support in-place scatter-gather. "
 				"Test Skipped.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	plaintext_len = ceil_byte_length(tdata->plaintext.len);
@@ -6043,17 +6043,17 @@ test_zuc_authentication(const struct wireless_test_data *tdata)
 	if (!(feat_flags & RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA) &&
 			(tdata->validAuthLenInBits.len % 8 != 0)) {
 		printf("Device doesn't support NON-Byte Aligned Data.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Check if device supports ZUC EIA3 */
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
@@ -6061,7 +6061,7 @@ test_zuc_authentication(const struct wireless_test_data *tdata)
 
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create ZUC session */
 	retval = create_wireless_algo_hash_session(ts_params->valid_devs[0],
@@ -6140,7 +6140,7 @@ test_zuc_auth_cipher(const struct wireless_test_data *tdata,
 
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -6148,27 +6148,27 @@ test_zuc_auth_cipher(const struct wireless_test_data *tdata,
 
 	if (!(feat_flags & RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED)) {
 		printf("Device doesn't support digest encrypted.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 	if (op_mode == IN_PLACE) {
 		if (!(feat_flags & RTE_CRYPTODEV_FF_IN_PLACE_SGL)) {
 			printf("Device doesn't support in-place scatter-gather "
 					"in both input and output mbufs.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 
 		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 			printf("Device doesn't support RAW data-path APIs.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 	} else {
 		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 	}
 
@@ -6338,7 +6338,7 @@ test_zuc_auth_cipher_sgl(const struct wireless_test_data *tdata,
 
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -6348,25 +6348,25 @@ test_zuc_auth_cipher_sgl(const struct wireless_test_data *tdata,
 		if (!(feat_flags & RTE_CRYPTODEV_FF_IN_PLACE_SGL)) {
 			printf("Device doesn't support in-place scatter-gather "
 					"in both input and output mbufs.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 
 		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 			printf("Device doesn't support RAW data-path APIs.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 	} else {
 		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 		if (!(feat_flags & RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED)) {
 			printf("Device doesn't support digest encrypted.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 	}
 
@@ -6586,7 +6586,7 @@ test_kasumi_decryption_test_case_3(void)
 {
 	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
 	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	return test_kasumi_decryption(&kasumi_test_case_3);
 }
 
@@ -6737,7 +6737,7 @@ test_snow3g_decryption_with_digest_test_case_1(void)
 
 	if (!(feat_flags & RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED)) {
 		printf("Device doesn't support encrypted digest operations.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	/*
@@ -6798,7 +6798,7 @@ test_snow3g_auth_cipher_test_case_3_sgl(void)
 {
 	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
 	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_test_case_3, IN_PLACE, 0);
 }
@@ -6815,7 +6815,7 @@ test_snow3g_auth_cipher_part_digest_enc_sgl(void)
 {
 	/* rte_crypto_mbuf_to_vec does not support incomplete mbuf build */
 	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	return test_snow3g_auth_cipher_sgl(
 		&snow3g_auth_cipher_partial_digest_encryption,
 			IN_PLACE, 0);
@@ -7142,13 +7142,13 @@ test_mixed_check_if_unsupported(const struct mixed_cipher_auth_test_data *tdata)
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = tdata->cipher_algo;
 	if (rte_cryptodev_sym_capability_get(dev_id, &cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Check if device supports particular hash algorithm */
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH;
 	cap_idx.algo.auth = tdata->auth_algo;
 	if (rte_cryptodev_sym_capability_get(dev_id, &cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	return 0;
 }
@@ -7173,9 +7173,9 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 
 	/* Check if device supports particular algorithms separately */
 	if (test_mixed_check_if_unsupported(tdata))
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -7183,7 +7183,7 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 
 	if (!(feat_flags & RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED)) {
 		printf("Device doesn't support digest encrypted.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	/* Create the session */
@@ -7270,7 +7270,7 @@ test_mixed_auth_cipher(const struct mixed_cipher_auth_test_data *tdata,
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 	ut_params->op = op;
 
@@ -7367,9 +7367,9 @@ test_mixed_auth_cipher_sgl(const struct mixed_cipher_auth_test_data *tdata,
 
 	/* Check if device supports particular algorithms */
 	if (test_mixed_check_if_unsupported(tdata))
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -7379,17 +7379,17 @@ test_mixed_auth_cipher_sgl(const struct mixed_cipher_auth_test_data *tdata,
 		if (!(feat_flags & RTE_CRYPTODEV_FF_IN_PLACE_SGL)) {
 			printf("Device doesn't support in-place scatter-gather "
 					"in both input and output mbufs.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 	} else {
 		if (!(feat_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)) {
 			printf("Device doesn't support out-of-place scatter-gather "
 					"in both input and output mbufs.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 		if (!(feat_flags & RTE_CRYPTODEV_FF_DIGEST_ENCRYPTED)) {
 			printf("Device doesn't support digest encrypted.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 	}
 
@@ -7476,7 +7476,7 @@ test_mixed_auth_cipher_sgl(const struct mixed_cipher_auth_test_data *tdata,
 			RTE_CRYPTO_OP_STATUS_INVALID_SESSION) {
 		printf("Device doesn't support this mixed combination. "
 				"Test Skipped.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 	ut_params->op = op;
 
@@ -8028,7 +8028,7 @@ test_authenticated_encryption(const struct aead_test_data *tdata)
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	/* Verify the capabilities */
@@ -8039,11 +8039,11 @@ test_authenticated_encryption(const struct aead_test_data *tdata)
 	capability = rte_cryptodev_sym_capability_get(
 			ts_params->valid_devs[0], &cap_idx);
 	if (capability == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	if (rte_cryptodev_sym_capability_check_aead(
 			capability, tdata->key.len, tdata->auth_tag.len,
 			tdata->aad.len, tdata->iv.len))
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create AEAD session */
 	retval = create_aead_session(ts_params->valid_devs[0],
@@ -8188,7 +8188,7 @@ static int test_pdcp_proto(int i, int oop, enum rte_crypto_cipher_operation opc,
 	sec_cap_idx.protocol = RTE_SECURITY_PROTOCOL_PDCP;
 	sec_cap_idx.pdcp.domain = domain;
 	if (rte_security_capability_get(ctx, &sec_cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Generate test mbuf data */
 	ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool);
@@ -8374,7 +8374,7 @@ test_pdcp_proto_SGL(int i, int oop,
 	sec_cap_idx.protocol = RTE_SECURITY_PROTOCOL_PDCP;
 	sec_cap_idx.pdcp.domain = pdcp_test_params[i].domain;
 	if (rte_security_capability_get(ctx, &sec_cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (fragsz > input_vec_len)
 		fragsz = input_vec_len;
@@ -8880,7 +8880,7 @@ test_PDCP_PROTO_all(void)
 	uint64_t feat_flags = dev_info.feature_flags;
 
 	if (!(feat_flags & RTE_CRYPTODEV_FF_SECURITY))
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Set action type */
 	ut_params->type = gbl_action_type == RTE_SECURITY_ACTION_TYPE_NONE ?
@@ -8889,7 +8889,7 @@ test_PDCP_PROTO_all(void)
 
 	if (security_proto_supported(ut_params->type,
 			RTE_SECURITY_PROTOCOL_PDCP) < 0)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	status = test_PDCP_PROTO_cplane_encap_all();
 	status += test_PDCP_PROTO_cplane_decap_all();
@@ -8936,7 +8936,7 @@ test_docsis_proto_uplink(int i, struct docsis_test_data *d_td)
 
 	sec_cap = rte_security_capability_get(ctx, &sec_cap_idx);
 	if (sec_cap == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	while ((crypto_cap = &sec_cap->crypto_capabilities[j++])->op !=
 			RTE_CRYPTO_OP_TYPE_UNDEFINED) {
@@ -8954,7 +8954,7 @@ test_docsis_proto_uplink(int i, struct docsis_test_data *d_td)
 	}
 
 	if (crypto_cap->op == RTE_CRYPTO_OP_TYPE_UNDEFINED)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Setup source mbuf payload */
 	ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool);
@@ -9112,7 +9112,7 @@ test_docsis_proto_downlink(int i, struct docsis_test_data *d_td)
 
 	sec_cap = rte_security_capability_get(ctx, &sec_cap_idx);
 	if (sec_cap == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	while ((crypto_cap = &sec_cap->crypto_capabilities[j++])->op !=
 			RTE_CRYPTO_OP_TYPE_UNDEFINED) {
@@ -9130,7 +9130,7 @@ test_docsis_proto_downlink(int i, struct docsis_test_data *d_td)
 	}
 
 	if (crypto_cap->op == RTE_CRYPTO_OP_TYPE_UNDEFINED)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Setup source mbuf payload */
 	ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool);
@@ -9265,10 +9265,10 @@ test_docsis_proto_downlink(int i, struct docsis_test_data *d_td)
 		printf("\t%2d)", n++);			\
 		printf("+++++ PASSED:" #func"\n");	\
 		p++;					\
-	} else if (ret == -ENOTSUP) {			\
+	} else if (ret == TEST_SKIPPED) {		\
 		printf("\t%2d)", n++);			\
-		printf("~~~~~ UNSUPP:" #func"\n");	\
-		u++;					\
+		printf("~~~~~ SKIPPED:" #func"\n");	\
+		s++;					\
 	} else {					\
 		printf("\t%2d)", n++);			\
 		printf("----- FAILED:" #func"\n");	\
@@ -9279,7 +9279,7 @@ test_docsis_proto_downlink(int i, struct docsis_test_data *d_td)
 static int
 test_DOCSIS_PROTO_uplink_all(void)
 {
-	int p = 0, u = 0, f = 0, n = 0;
+	int p = 0, s = 0, f = 0, n = 0;
 
 	TEST_DOCSIS_COUNT(test_docsis_proto_uplink(1, &docsis_test_case_1));
 	TEST_DOCSIS_COUNT(test_docsis_proto_uplink(2, &docsis_test_case_2));
@@ -9309,8 +9309,8 @@ test_DOCSIS_PROTO_uplink_all(void)
 	TEST_DOCSIS_COUNT(test_docsis_proto_uplink(26, &docsis_test_case_26));
 
 	if (f)
-		printf("## %s: %d passed out of %d (%d unsupported)\n",
-			__func__, p, n, u);
+		printf("## %s: %d passed out of %d (%d skipped)\n",
+			__func__, p, n, s);
 
 	return f;
 };
@@ -9318,7 +9318,7 @@ test_DOCSIS_PROTO_uplink_all(void)
 static int
 test_DOCSIS_PROTO_downlink_all(void)
 {
-	int p = 0, u = 0, f = 0, n = 0;
+	int p = 0, s = 0, f = 0, n = 0;
 
 	TEST_DOCSIS_COUNT(test_docsis_proto_downlink(1, &docsis_test_case_1));
 	TEST_DOCSIS_COUNT(test_docsis_proto_downlink(2, &docsis_test_case_2));
@@ -9348,8 +9348,8 @@ test_DOCSIS_PROTO_downlink_all(void)
 	TEST_DOCSIS_COUNT(test_docsis_proto_downlink(26, &docsis_test_case_26));
 
 	if (f)
-		printf("## %s: %d passed out of %d (%d unsupported)\n",
-			__func__, p, n, u);
+		printf("## %s: %d passed out of %d (%d skipped)\n",
+			__func__, p, n, s);
 
 	return f;
 };
@@ -9366,7 +9366,7 @@ test_DOCSIS_PROTO_all(void)
 	uint64_t feat_flags = dev_info.feature_flags;
 
 	if (!(feat_flags & RTE_CRYPTODEV_FF_SECURITY))
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Set action type */
 	ut_params->type = gbl_action_type == RTE_SECURITY_ACTION_TYPE_NONE ?
@@ -9375,7 +9375,7 @@ test_DOCSIS_PROTO_all(void)
 
 	if (security_proto_supported(ut_params->type,
 			RTE_SECURITY_PROTOCOL_DOCSIS) < 0)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	status = test_DOCSIS_PROTO_uplink_all();
 	status += test_DOCSIS_PROTO_downlink_all();
@@ -9547,7 +9547,7 @@ test_AES_GCM_auth_encryption_fail_iv_corrupt(void)
 	memcpy(&tdata, &gcm_test_case_7, sizeof(struct aead_test_data));
 	tdata.iv.data[0] += 1;
 	res = test_authenticated_encryption(&tdata);
-	if (res == -ENOTSUP)
+	if (res == TEST_SKIPPED)
 		return res;
 	TEST_ASSERT_EQUAL(res, TEST_FAILED, "encryption not failed");
 	return TEST_SUCCESS;
@@ -9563,7 +9563,7 @@ test_AES_GCM_auth_encryption_fail_in_data_corrupt(void)
 	memcpy(&tdata, &gcm_test_case_7, sizeof(struct aead_test_data));
 	tdata.plaintext.data[0] += 1;
 	res = test_authenticated_encryption(&tdata);
-	if (res == -ENOTSUP)
+	if (res == TEST_SKIPPED)
 		return res;
 	TEST_ASSERT_EQUAL(res, TEST_FAILED, "encryption not failed");
 	return TEST_SUCCESS;
@@ -9579,7 +9579,7 @@ test_AES_GCM_auth_encryption_fail_out_data_corrupt(void)
 	memcpy(&tdata, &gcm_test_case_7, sizeof(struct aead_test_data));
 	tdata.ciphertext.data[0] += 1;
 	res = test_authenticated_encryption(&tdata);
-	if (res == -ENOTSUP)
+	if (res == TEST_SKIPPED)
 		return res;
 	TEST_ASSERT_EQUAL(res, TEST_FAILED, "encryption not failed");
 	return TEST_SUCCESS;
@@ -9595,7 +9595,7 @@ test_AES_GCM_auth_encryption_fail_aad_len_corrupt(void)
 	memcpy(&tdata, &gcm_test_case_7, sizeof(struct aead_test_data));
 	tdata.aad.len += 1;
 	res = test_authenticated_encryption(&tdata);
-	if (res == -ENOTSUP)
+	if (res == TEST_SKIPPED)
 		return res;
 	TEST_ASSERT_EQUAL(res, TEST_FAILED, "encryption not failed");
 	return TEST_SUCCESS;
@@ -9614,7 +9614,7 @@ test_AES_GCM_auth_encryption_fail_aad_corrupt(void)
 	aad[0] += 1;
 	tdata.aad.data = aad;
 	res = test_authenticated_encryption(&tdata);
-	if (res == -ENOTSUP)
+	if (res == TEST_SKIPPED)
 		return res;
 	TEST_ASSERT_EQUAL(res, TEST_FAILED, "encryption not failed");
 	return TEST_SUCCESS;
@@ -9630,7 +9630,7 @@ test_AES_GCM_auth_encryption_fail_tag_corrupt(void)
 	memcpy(&tdata, &gcm_test_case_7, sizeof(struct aead_test_data));
 	tdata.auth_tag.data[0] += 1;
 	res = test_authenticated_encryption(&tdata);
-	if (res == -ENOTSUP)
+	if (res == TEST_SKIPPED)
 		return res;
 	TEST_ASSERT_EQUAL(res, TEST_FAILED, "encryption not failed");
 	return TEST_SUCCESS;
@@ -9653,7 +9653,7 @@ test_authenticated_decryption(const struct aead_test_data *tdata)
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	/* Verify the capabilities */
@@ -9664,11 +9664,11 @@ test_authenticated_decryption(const struct aead_test_data *tdata)
 	capability = rte_cryptodev_sym_capability_get(
 			ts_params->valid_devs[0], &cap_idx);
 	if (capability == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	if (rte_cryptodev_sym_capability_check_aead(
 			capability, tdata->key.len, tdata->auth_tag.len,
 			tdata->aad.len, tdata->iv.len))
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create AEAD session */
 	retval = create_aead_session(ts_params->valid_devs[0],
@@ -9898,7 +9898,7 @@ test_AES_GCM_auth_decryption_fail_iv_corrupt(void)
 	memcpy(&tdata, &gcm_test_case_7, sizeof(struct aead_test_data));
 	tdata.iv.data[0] += 1;
 	res = test_authenticated_decryption(&tdata);
-	if (res == -ENOTSUP)
+	if (res == TEST_SKIPPED)
 		return res;
 	TEST_ASSERT_EQUAL(res, TEST_FAILED, "decryption not failed");
 	return TEST_SUCCESS;
@@ -9914,7 +9914,7 @@ test_AES_GCM_auth_decryption_fail_in_data_corrupt(void)
 	memcpy(&tdata, &gcm_test_case_7, sizeof(struct aead_test_data));
 	tdata.plaintext.data[0] += 1;
 	res = test_authenticated_decryption(&tdata);
-	if (res == -ENOTSUP)
+	if (res == TEST_SKIPPED)
 		return res;
 	TEST_ASSERT_EQUAL(res, TEST_FAILED, "decryption not failed");
 	return TEST_SUCCESS;
@@ -9929,7 +9929,7 @@ test_AES_GCM_auth_decryption_fail_out_data_corrupt(void)
 	memcpy(&tdata, &gcm_test_case_7, sizeof(struct aead_test_data));
 	tdata.ciphertext.data[0] += 1;
 	res = test_authenticated_decryption(&tdata);
-	if (res == -ENOTSUP)
+	if (res == TEST_SKIPPED)
 		return res;
 	TEST_ASSERT_EQUAL(res, TEST_FAILED, "decryption not failed");
 	return TEST_SUCCESS;
@@ -9944,7 +9944,7 @@ test_AES_GCM_auth_decryption_fail_aad_len_corrupt(void)
 	memcpy(&tdata, &gcm_test_case_7, sizeof(struct aead_test_data));
 	tdata.aad.len += 1;
 	res = test_authenticated_decryption(&tdata);
-	if (res == -ENOTSUP)
+	if (res == TEST_SKIPPED)
 		return res;
 	TEST_ASSERT_EQUAL(res, TEST_FAILED, "decryption not failed");
 	return TEST_SUCCESS;
@@ -9962,7 +9962,7 @@ test_AES_GCM_auth_decryption_fail_aad_corrupt(void)
 	aad[0] += 1;
 	tdata.aad.data = aad;
 	res = test_authenticated_decryption(&tdata);
-	if (res == -ENOTSUP)
+	if (res == TEST_SKIPPED)
 		return res;
 	TEST_ASSERT_EQUAL(res, TEST_FAILED, "decryption not failed");
 	return TEST_SUCCESS;
@@ -9977,7 +9977,7 @@ test_AES_GCM_auth_decryption_fail_tag_corrupt(void)
 	memcpy(&tdata, &gcm_test_case_7, sizeof(struct aead_test_data));
 	tdata.auth_tag.data[0] += 1;
 	res = test_authenticated_decryption(&tdata);
-	if (res == -ENOTSUP)
+	if (res == TEST_SKIPPED)
 		return res;
 	TEST_ASSERT_EQUAL(res, TEST_FAILED, "authentication not failed");
 	return TEST_SUCCESS;
@@ -9999,14 +9999,14 @@ test_authenticated_encryption_oop(const struct aead_test_data *tdata)
 	cap_idx.algo.aead = tdata->algo;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* not supported with CPU crypto */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create AEAD session */
 	retval = create_aead_session(ts_params->valid_devs[0],
@@ -10091,12 +10091,12 @@ test_authenticated_decryption_oop(const struct aead_test_data *tdata)
 	cap_idx.algo.aead = tdata->algo;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* not supported with CPU crypto and raw data-path APIs*/
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO ||
 			global_api_test_type == CRYPTODEV_RAW_API_TEST)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create AEAD session */
 	retval = create_aead_session(ts_params->valid_devs[0],
@@ -10176,12 +10176,12 @@ test_authenticated_encryption_sessionless(
 
 	if (!(feat_flags & RTE_CRYPTODEV_FF_SYM_SESSIONLESS)) {
 		printf("Device doesn't support Sessionless ops.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	/* not supported with CPU crypto */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -10189,7 +10189,7 @@ test_authenticated_encryption_sessionless(
 	cap_idx.algo.aead = tdata->algo;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool);
 
@@ -10278,18 +10278,18 @@ test_authenticated_decryption_sessionless(
 
 	if (!(feat_flags & RTE_CRYPTODEV_FF_SYM_SESSIONLESS)) {
 		printf("Device doesn't support Sessionless ops.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	/* not supported with CPU crypto */
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -10297,7 +10297,7 @@ test_authenticated_decryption_sessionless(
 	cap_idx.algo.aead = tdata->algo;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* alloc mbuf and set payload */
 	ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool);
@@ -10481,7 +10481,7 @@ test_stats(void)
 	struct rte_cryptodev_stats stats;
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Verify the capabilities */
 	struct rte_cryptodev_sym_capability_idx cap_idx;
@@ -10489,16 +10489,16 @@ test_stats(void)
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_SHA1_HMAC;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_AES_CBC;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (rte_cryptodev_stats_get(ts_params->valid_devs[0], &stats)
-			== -ENOTSUP)
-		return -ENOTSUP;
+			== TEST_SKIPPED)
+		return TEST_SKIPPED;
 
 	rte_cryptodev_stats_reset(ts_params->valid_devs[0]);
 	TEST_ASSERT((rte_cryptodev_stats_get(ts_params->valid_devs[0] + 600,
@@ -10632,7 +10632,7 @@ test_MD5_HMAC_generate(const struct HMAC_MD5_vector *test_case)
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	/* Verify the capabilities */
@@ -10641,7 +10641,7 @@ test_MD5_HMAC_generate(const struct HMAC_MD5_vector *test_case)
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_MD5_HMAC;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (MD5_HMAC_create_session(ts_params, ut_params,
 			RTE_CRYPTO_AUTH_OP_GENERATE, test_case))
@@ -10705,7 +10705,7 @@ test_MD5_HMAC_verify(const struct HMAC_MD5_vector *test_case)
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	/* Verify the capabilities */
@@ -10714,7 +10714,7 @@ test_MD5_HMAC_verify(const struct HMAC_MD5_vector *test_case)
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_MD5_HMAC;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (MD5_HMAC_create_session(ts_params, ut_params,
 			RTE_CRYPTO_AUTH_OP_VERIFY, test_case)) {
@@ -10789,12 +10789,12 @@ test_multi_session(void)
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_SHA512_HMAC;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_AES_CBC;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	test_AES_CBC_HMAC_SHA512_decrypt_create_session_params(ut_params,
 			aes_cbc_key, hmac_sha512_key);
@@ -10917,12 +10917,12 @@ test_multi_session_random_usage(void)
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_SHA512_HMAC;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = RTE_CRYPTO_CIPHER_AES_CBC;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
 
@@ -11015,7 +11015,7 @@ test_null_invalid_operation(void)
 	/* This test is for NULL PMD only */
 	if (gbl_driver_id != rte_cryptodev_driver_id_get(
 			RTE_STR(CRYPTODEV_NAME_NULL_PMD)))
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Setup Cipher Parameters */
 	ut_params->cipher_xform.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
@@ -11072,7 +11072,7 @@ test_null_burst_operation(void)
 	/* This test is for NULL PMD only */
 	if (gbl_driver_id != rte_cryptodev_driver_id_get(
 			RTE_STR(CRYPTODEV_NAME_NULL_PMD)))
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Setup Cipher Parameters */
 	ut_params->cipher_xform.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
@@ -11536,7 +11536,7 @@ test_AES_GMAC_authentication(const struct gmac_test_data *tdata)
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	int retval;
@@ -11553,7 +11553,7 @@ test_AES_GMAC_authentication(const struct gmac_test_data *tdata)
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_AES_GMAC;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	retval = create_gmac_session(ts_params->valid_devs[0],
 			tdata, RTE_CRYPTO_AUTH_OP_GENERATE);
@@ -11670,7 +11670,7 @@ test_AES_GMAC_authentication_verify(const struct gmac_test_data *tdata)
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	TEST_ASSERT_NOT_EQUAL(tdata->gmac_tag.len, 0,
@@ -11682,7 +11682,7 @@ test_AES_GMAC_authentication_verify(const struct gmac_test_data *tdata)
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_AES_GMAC;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	retval = create_gmac_session(ts_params->valid_devs[0],
 			tdata, RTE_CRYPTO_AUTH_OP_VERIFY);
@@ -11795,7 +11795,7 @@ test_AES_GMAC_authentication_SGL(const struct gmac_test_data *tdata,
 	cap_idx.algo.auth = RTE_CRYPTO_AUTH_AES_GMAC;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Check for any input SGL support */
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -11804,7 +11804,7 @@ test_AES_GMAC_authentication_SGL(const struct gmac_test_data *tdata,
 	if ((!(feature_flags & RTE_CRYPTODEV_FF_IN_PLACE_SGL)) ||
 			(!(feature_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT)) ||
 			(!(feature_flags & RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT)))
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	if (fragsz > tdata->plaintext.len)
 		fragsz = tdata->plaintext.len;
@@ -11884,7 +11884,7 @@ test_AES_GMAC_authentication_SGL(const struct gmac_test_data *tdata,
 	ut_params->op->sym->m_src = ut_params->ibuf;
 
 	if (gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	TEST_ASSERT_NOT_NULL(
 		process_crypto_request(ts_params->valid_devs[0],
@@ -12417,7 +12417,7 @@ test_authentication_verify_fail_when_data_corruption(
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	/* Verify the capabilities */
@@ -12426,7 +12426,7 @@ test_authentication_verify_fail_when_data_corruption(
 	cap_idx.algo.auth = reference->auth_algo;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 
 	/* Create session */
@@ -12499,7 +12499,7 @@ test_authentication_verify_GMAC_fail_when_corruption(
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	/* Verify the capabilities */
@@ -12508,7 +12508,7 @@ test_authentication_verify_GMAC_fail_when_corruption(
 	cap_idx.algo.auth = reference->auth_algo;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create session */
 	retval = create_auth_cipher_session(ut_params,
@@ -12584,7 +12584,7 @@ test_authenticated_decryption_fail_when_corruption(
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	/* Verify the capabilities */
@@ -12593,12 +12593,12 @@ test_authenticated_decryption_fail_when_corruption(
 	cap_idx.algo.auth = reference->auth_algo;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = reference->crypto_algo;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create session */
 	retval = create_auth_cipher_session(ut_params,
@@ -12674,7 +12674,7 @@ test_authenticated_encryt_with_esn(
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	/* Verify the capabilities */
@@ -12683,12 +12683,12 @@ test_authenticated_encryt_with_esn(
 	cap_idx.algo.auth = reference->auth_algo;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = reference->crypto_algo;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create session */
 	memcpy(cipher_key, reference->cipher_key.data,
@@ -12807,7 +12807,7 @@ test_authenticated_decrypt_with_esn(
 	if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 		printf("Device doesn't support RAW data-path APIs.\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	/* Verify the capabilities */
@@ -12816,12 +12816,12 @@ test_authenticated_decrypt_with_esn(
 	cap_idx.algo.auth = reference->auth_algo;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER;
 	cap_idx.algo.cipher = reference->crypto_algo;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Create session */
 	memcpy(cipher_key, reference->cipher_key.data,
@@ -13013,11 +13013,11 @@ test_authenticated_encryption_SGL(const struct aead_test_data *tdata,
 	cap_idx.algo.aead = tdata->algo;
 	if (rte_cryptodev_sym_capability_get(ts_params->valid_devs[0],
 			&cap_idx) == NULL)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* OOP not supported with CPU crypto */
 	if (oop && gbl_action_type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	/* Detailed check for the particular SGL support flag */
 	rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info);
@@ -13025,14 +13025,14 @@ test_authenticated_encryption_SGL(const struct aead_test_data *tdata,
 		unsigned int sgl_in = fragsz < tdata->plaintext.len;
 		if (sgl_in && (!(dev_info.feature_flags &
 				RTE_CRYPTODEV_FF_IN_PLACE_SGL)))
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 
 		uint64_t feat_flags = dev_info.feature_flags;
 
 		if ((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
 			(!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
 			printf("Device doesn't support RAW data-path APIs.\n");
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		}
 	} else {
 		unsigned int sgl_in = fragsz < tdata->plaintext.len;
@@ -13040,19 +13040,19 @@ test_authenticated_encryption_SGL(const struct aead_test_data *tdata,
 				tdata->plaintext.len;
 		/* Raw data path API does not support OOP */
 		if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-			return -ENOTSUP;
+			return TEST_SKIPPED;
 		if (sgl_in && !sgl_out) {
 			if (!(dev_info.feature_flags &
 					RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT))
-				return -ENOTSUP;
+				return TEST_SKIPPED;
 		} else if (!sgl_in && sgl_out) {
 			if (!(dev_info.feature_flags &
 					RTE_CRYPTODEV_FF_OOP_LB_IN_SGL_OUT))
-				return -ENOTSUP;
+				return TEST_SKIPPED;
 		} else if (sgl_in && sgl_out) {
 			if (!(dev_info.feature_flags &
 					RTE_CRYPTODEV_FF_OOP_SGL_IN_SGL_OUT))
-				return -ENOTSUP;
+				return TEST_SKIPPED;
 		}
 	}
 
@@ -13323,7 +13323,7 @@ test_AES_GCM_auth_encrypt_SGL_in_place_1500B(void)
 	/* This test is not for OPENSSL PMD */
 	if (gbl_driver_id == rte_cryptodev_driver_id_get(
 			RTE_STR(CRYPTODEV_NAME_OPENSSL_PMD)))
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 
 	return test_authenticated_encryption_SGL(
 			&gcm_test_case_SGL_1, IN_PLACE, 1500, 0);
diff --git a/app/test/test_cryptodev_asym.c b/app/test/test_cryptodev_asym.c
index 85cd076059..e935f38ca9 100644
--- a/app/test/test_cryptodev_asym.c
+++ b/app/test/test_cryptodev_asym.c
@@ -327,7 +327,7 @@ test_cryptodev_asym_op(struct crypto_testsuite_params *ts_params,
 	if (capability == NULL) {
 		RTE_LOG(INFO, USER1,
 			"Device doesn't support MODEX. Test Skipped\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	/* Generate crypto op data structure */
@@ -665,7 +665,7 @@ test_rsa_sign_verify(void)
 				RTE_CRYPTODEV_FF_RSA_PRIV_OP_KEY_EXP)) {
 		RTE_LOG(INFO, USER1, "Device doesn't support sign op with "
 			"exponent key type. Test Skipped\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	sess = rte_cryptodev_asym_session_create(sess_mpool);
@@ -714,7 +714,7 @@ test_rsa_enc_dec(void)
 				RTE_CRYPTODEV_FF_RSA_PRIV_OP_KEY_EXP)) {
 		RTE_LOG(INFO, USER1, "Device doesn't support decrypt op with "
 			"exponent key type. Test skipped\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	sess = rte_cryptodev_asym_session_create(sess_mpool);
@@ -761,7 +761,7 @@ test_rsa_sign_verify_crt(void)
 	if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_RSA_PRIV_OP_KEY_QT)) {
 		RTE_LOG(INFO, USER1, "Device doesn't support sign op with "
 			"quintuple key type. Test skipped\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	sess = rte_cryptodev_asym_session_create(sess_mpool);
@@ -809,7 +809,7 @@ test_rsa_enc_dec_crt(void)
 	if (!(dev_info.feature_flags & RTE_CRYPTODEV_FF_RSA_PRIV_OP_KEY_QT)) {
 		RTE_LOG(INFO, USER1, "Device doesn't support decrypt op with "
 			"quintuple key type. Test skipped\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	sess = rte_cryptodev_asym_session_create(sess_mpool);
@@ -1501,7 +1501,7 @@ test_mod_inv(void)
 	if (capability == NULL) {
 		RTE_LOG(INFO, USER1,
 			"Device doesn't support MOD INV. Test Skipped\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if (rte_cryptodev_asym_xform_capability_check_modlen(
@@ -1509,7 +1509,7 @@ test_mod_inv(void)
 		modinv_xform.modinv.modulus.length)) {
 		RTE_LOG(ERR, USER1,
 				 "Invalid MODULUS length specified\n");
-				return -ENOTSUP;
+				return TEST_SKIPPED;
 		}
 
 	sess = rte_cryptodev_asym_session_create(sess_mpool);
@@ -1626,14 +1626,14 @@ test_mod_exp(void)
 	if (capability == NULL) {
 		RTE_LOG(INFO, USER1,
 			"Device doesn't support MOD EXP. Test Skipped\n");
-		return -ENOTSUP;
+		return TEST_SKIPPED;
 	}
 
 	if (rte_cryptodev_asym_xform_capability_check_modlen(
 			capability, modex_xform.modex.modulus.length)) {
 		RTE_LOG(ERR, USER1,
 				"Invalid MODULUS length specified\n");
-				return -ENOTSUP;
+				return TEST_SKIPPED;
 		}
 
 	/* generate crypto op data structure */
-- 
2.25.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* [dpdk-dev] [PATCH v3 5/7] test/crypto: move testsuite params to header file
  2021-04-23 16:18 [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework Ciara Power
                   ` (3 preceding siblings ...)
  2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 4/7] test/crypto: replace unsupported with skipped Ciara Power
@ 2021-04-23 16:18 ` Ciara Power
  2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 6/7] test/crypto: fix return value on test skipped Ciara Power
                   ` (4 subsequent siblings)
  9 siblings, 0 replies; 15+ messages in thread
From: Ciara Power @ 2021-04-23 16:18 UTC (permalink / raw)
  To: dev
  Cc: thomas, declan.doherty, gakhil, aconole, hemant.agrawal, anoobj,
	ruifeng.wang, asomalap, ajit.khaparde, g.singh, Ciara Power,
	Konstantin Ananyev, Bernard Iremonger, Vladimir Medvedkin

The testsuite params struct and ut functions are now in the cryptodev
test header file. This will allow them be used outside of the
cryptodev_test.c file. They will be used in a subsequent patch by the
blockcipher test.

As a result of this change, slight renaming changes were necessary
for ipsec and asym tests, to avoid a clash in names.

Signed-off-by: Ciara Power <ciara.power@intel.com>
---
 app/test/test_cryptodev.c      | 18 ++-----
 app/test/test_cryptodev.h      | 20 ++++++++
 app/test/test_cryptodev_asym.c | 93 ++++++++++++++++++----------------
 app/test/test_ipsec.c          | 32 ++++++------
 4 files changed, 89 insertions(+), 74 deletions(-)

diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index 2c9477680e..67adfedd72 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -72,19 +72,6 @@ static enum rte_security_session_action_type gbl_action_type =
 
 enum cryptodev_api_test_type global_api_test_type = CRYPTODEV_API_TEST;
 
-struct crypto_testsuite_params {
-	struct rte_mempool *mbuf_pool;
-	struct rte_mempool *large_mbuf_pool;
-	struct rte_mempool *op_mpool;
-	struct rte_mempool *session_mpool;
-	struct rte_mempool *session_priv_mpool;
-	struct rte_cryptodev_config conf;
-	struct rte_cryptodev_qp_conf qp_conf;
-
-	uint8_t valid_devs[RTE_CRYPTO_MAX_DEVS];
-	uint8_t valid_dev_count;
-};
-
 struct crypto_unittest_params {
 	struct rte_crypto_sym_xform cipher_xform;
 	struct rte_crypto_sym_xform auth_xform;
@@ -486,6 +473,7 @@ process_crypto_request(uint8_t dev_id, struct rte_crypto_op *op)
 }
 
 static struct crypto_testsuite_params testsuite_params = { NULL };
+struct crypto_testsuite_params *p_testsuite_params = &testsuite_params;
 static struct crypto_unittest_params unittest_params;
 
 static int
@@ -1313,7 +1301,7 @@ dev_configure_and_start(uint64_t ff_disable)
 	return TEST_SUCCESS;
 }
 
-static int
+int
 ut_setup(void)
 {
 	/* Configure and start the device with security feature disabled */
@@ -1327,7 +1315,7 @@ ut_setup_security(void)
 	return dev_configure_and_start(0);
 }
 
-static void
+void
 ut_teardown(void)
 {
 	struct crypto_testsuite_params *ts_params = &testsuite_params;
diff --git a/app/test/test_cryptodev.h b/app/test/test_cryptodev.h
index 5c41e36f44..f81f8e372f 100644
--- a/app/test/test_cryptodev.h
+++ b/app/test/test_cryptodev.h
@@ -79,6 +79,20 @@ enum cryptodev_api_test_type {
 
 extern enum cryptodev_api_test_type global_api_test_type;
 
+extern struct crypto_testsuite_params *p_testsuite_params;
+struct crypto_testsuite_params {
+	struct rte_mempool *mbuf_pool;
+	struct rte_mempool *large_mbuf_pool;
+	struct rte_mempool *op_mpool;
+	struct rte_mempool *session_mpool;
+	struct rte_mempool *session_priv_mpool;
+	struct rte_cryptodev_config conf;
+	struct rte_cryptodev_qp_conf qp_conf;
+
+	uint8_t valid_devs[RTE_CRYPTO_MAX_DEVS];
+	uint8_t valid_dev_count;
+};
+
 /**
  * Write (spread) data from buffer to mbuf data
  *
@@ -234,4 +248,10 @@ int
 check_aead_capabilities_supported(const enum rte_crypto_aead_algorithm *aeads,
 		uint16_t num_aeads);
 
+int
+ut_setup(void);
+
+void
+ut_teardown(void);
+
 #endif /* TEST_CRYPTODEV_H_ */
diff --git a/app/test/test_cryptodev_asym.c b/app/test/test_cryptodev_asym.c
index e935f38ca9..b36eec9abf 100644
--- a/app/test/test_cryptodev_asym.c
+++ b/app/test/test_cryptodev_asym.c
@@ -35,7 +35,7 @@
 #define TEST_VECTOR_SIZE 256
 
 static int gbl_driver_id;
-struct crypto_testsuite_params {
+struct crypto_testsuite_params_asym {
 	struct rte_mempool *op_mpool;
 	struct rte_mempool *session_mpool;
 	struct rte_cryptodev_config conf;
@@ -63,12 +63,12 @@ static struct test_cases_array test_vector = {0, { NULL } };
 
 static uint32_t test_index;
 
-static struct crypto_testsuite_params testsuite_params = { NULL };
+static struct crypto_testsuite_params_asym testsuite_params = { NULL };
 
 static int
 queue_ops_rsa_sign_verify(struct rte_cryptodev_asym_session *sess)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 	struct rte_mempool *op_mpool = ts_params->op_mpool;
 	uint8_t dev_id = ts_params->valid_devs[0];
 	struct rte_crypto_op *op, *result_op;
@@ -159,7 +159,7 @@ queue_ops_rsa_sign_verify(struct rte_cryptodev_asym_session *sess)
 static int
 queue_ops_rsa_enc_dec(struct rte_cryptodev_asym_session *sess)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 	struct rte_mempool *op_mpool = ts_params->op_mpool;
 	uint8_t dev_id = ts_params->valid_devs[0];
 	struct rte_crypto_op *op, *result_op;
@@ -300,7 +300,7 @@ test_cryptodev_asym_ver(struct rte_crypto_op *op,
 }
 
 static int
-test_cryptodev_asym_op(struct crypto_testsuite_params *ts_params,
+test_cryptodev_asym_op(struct crypto_testsuite_params_asym *ts_params,
 	union test_case_structure *data_tc,
 	char *test_msg, int sessionless, enum rte_crypto_asym_op_type type,
 	enum rte_crypto_rsa_priv_key_type key_type)
@@ -617,7 +617,7 @@ static int
 test_one_by_one(void)
 {
 	int status = TEST_SUCCESS;
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 	uint32_t i = 0;
 	uint8_t dev_id = ts_params->valid_devs[0];
 	struct rte_cryptodev_info dev_info;
@@ -650,7 +650,7 @@ test_one_by_one(void)
 static int
 test_rsa_sign_verify(void)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 	struct rte_mempool *sess_mpool = ts_params->session_mpool;
 	uint8_t dev_id = ts_params->valid_devs[0];
 	struct rte_cryptodev_asym_session *sess;
@@ -699,7 +699,7 @@ test_rsa_sign_verify(void)
 static int
 test_rsa_enc_dec(void)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 	struct rte_mempool *sess_mpool = ts_params->session_mpool;
 	uint8_t dev_id = ts_params->valid_devs[0];
 	struct rte_cryptodev_asym_session *sess;
@@ -747,7 +747,7 @@ test_rsa_enc_dec(void)
 static int
 test_rsa_sign_verify_crt(void)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 	struct rte_mempool *sess_mpool = ts_params->session_mpool;
 	uint8_t dev_id = ts_params->valid_devs[0];
 	struct rte_cryptodev_asym_session *sess;
@@ -795,7 +795,7 @@ test_rsa_sign_verify_crt(void)
 static int
 test_rsa_enc_dec_crt(void)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 	struct rte_mempool *sess_mpool = ts_params->session_mpool;
 	uint8_t dev_id = ts_params->valid_devs[0];
 	struct rte_cryptodev_asym_session *sess;
@@ -842,7 +842,7 @@ test_rsa_enc_dec_crt(void)
 static int
 testsuite_setup(void)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 	uint8_t valid_devs[RTE_CRYPTO_MAX_DEVS];
 	struct rte_cryptodev_info info;
 	int ret, dev_id = -1;
@@ -959,7 +959,7 @@ testsuite_setup(void)
 static void
 testsuite_teardown(void)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 
 	if (ts_params->op_mpool != NULL) {
 		RTE_LOG(DEBUG, USER1, "CRYPTO_OP_POOL count %u\n",
@@ -974,9 +974,9 @@ testsuite_teardown(void)
 }
 
 static int
-ut_setup(void)
+ut_setup_asym(void)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 
 	uint16_t qp_id;
 
@@ -1008,9 +1008,9 @@ ut_setup(void)
 }
 
 static void
-ut_teardown(void)
+ut_teardown_asym(void)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 	struct rte_cryptodev_stats stats;
 
 	rte_cryptodev_stats_get(ts_params->valid_devs[0], &stats);
@@ -1056,7 +1056,7 @@ static inline void print_asym_capa(
 static int
 test_capability(void)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 	uint8_t dev_id = ts_params->valid_devs[0];
 	struct rte_cryptodev_info dev_info;
 	const struct rte_cryptodev_capabilities *dev_capa;
@@ -1093,7 +1093,7 @@ test_capability(void)
 static int
 test_dh_gen_shared_sec(struct rte_crypto_asym_xform *xfrm)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 	struct rte_mempool *op_mpool = ts_params->op_mpool;
 	struct rte_mempool *sess_mpool = ts_params->session_mpool;
 	uint8_t dev_id = ts_params->valid_devs[0];
@@ -1186,7 +1186,7 @@ test_dh_gen_shared_sec(struct rte_crypto_asym_xform *xfrm)
 static int
 test_dh_gen_priv_key(struct rte_crypto_asym_xform *xfrm)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 	struct rte_mempool *op_mpool = ts_params->op_mpool;
 	struct rte_mempool *sess_mpool = ts_params->session_mpool;
 	uint8_t dev_id = ts_params->valid_devs[0];
@@ -1277,7 +1277,7 @@ test_dh_gen_priv_key(struct rte_crypto_asym_xform *xfrm)
 static int
 test_dh_gen_pub_key(struct rte_crypto_asym_xform *xfrm)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 	struct rte_mempool *op_mpool = ts_params->op_mpool;
 	struct rte_mempool *sess_mpool = ts_params->session_mpool;
 	uint8_t dev_id = ts_params->valid_devs[0];
@@ -1376,7 +1376,7 @@ test_dh_gen_pub_key(struct rte_crypto_asym_xform *xfrm)
 static int
 test_dh_gen_kp(struct rte_crypto_asym_xform *xfrm)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 	struct rte_mempool *op_mpool = ts_params->op_mpool;
 	struct rte_mempool *sess_mpool = ts_params->session_mpool;
 	uint8_t dev_id = ts_params->valid_devs[0];
@@ -1473,7 +1473,7 @@ test_dh_gen_kp(struct rte_crypto_asym_xform *xfrm)
 static int
 test_mod_inv(void)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 	struct rte_mempool *op_mpool = ts_params->op_mpool;
 	struct rte_mempool *sess_mpool = ts_params->session_mpool;
 	uint8_t dev_id = ts_params->valid_devs[0];
@@ -1597,7 +1597,7 @@ test_mod_inv(void)
 static int
 test_mod_exp(void)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 	struct rte_mempool *op_mpool = ts_params->op_mpool;
 	struct rte_mempool *sess_mpool = ts_params->session_mpool;
 	uint8_t dev_id = ts_params->valid_devs[0];
@@ -1757,7 +1757,7 @@ test_dh_keygenration(void)
 static int
 test_dsa_sign(void)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 	struct rte_mempool *op_mpool = ts_params->op_mpool;
 	struct rte_mempool *sess_mpool = ts_params->session_mpool;
 	uint8_t dev_id = ts_params->valid_devs[0];
@@ -1901,7 +1901,7 @@ test_dsa(void)
 static int
 test_ecdsa_sign_verify(enum curve curve_id)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 	struct rte_mempool *sess_mpool = ts_params->session_mpool;
 	struct rte_mempool *op_mpool = ts_params->op_mpool;
 	struct crypto_testsuite_ecdsa_params input_params;
@@ -2114,7 +2114,7 @@ test_ecdsa_sign_verify_all_curve(void)
 static int
 test_ecpm(enum curve curve_id)
 {
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
+	struct crypto_testsuite_params_asym *ts_params = &testsuite_params;
 	struct rte_mempool *sess_mpool = ts_params->session_mpool;
 	struct rte_mempool *op_mpool = ts_params->op_mpool;
 	struct crypto_testsuite_ecpm_params input_params;
@@ -2289,16 +2289,20 @@ static struct unit_test_suite cryptodev_openssl_asym_testsuite  = {
 	.setup = testsuite_setup,
 	.teardown = testsuite_teardown,
 	.unit_test_cases = {
-		TEST_CASE_ST(ut_setup, ut_teardown, test_capability),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_dsa),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_dh_keygenration),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_rsa_enc_dec),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_rsa_sign_verify),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_rsa_enc_dec_crt),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_rsa_sign_verify_crt),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_mod_inv),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_mod_exp),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_one_by_one),
+		TEST_CASE_ST(ut_setup_asym, ut_teardown_asym, test_capability),
+		TEST_CASE_ST(ut_setup_asym, ut_teardown_asym, test_dsa),
+		TEST_CASE_ST(ut_setup_asym, ut_teardown_asym,
+				test_dh_keygenration),
+		TEST_CASE_ST(ut_setup_asym, ut_teardown_asym, test_rsa_enc_dec),
+		TEST_CASE_ST(ut_setup_asym, ut_teardown_asym,
+				test_rsa_sign_verify),
+		TEST_CASE_ST(ut_setup_asym, ut_teardown_asym,
+				test_rsa_enc_dec_crt),
+		TEST_CASE_ST(ut_setup_asym, ut_teardown_asym,
+				test_rsa_sign_verify_crt),
+		TEST_CASE_ST(ut_setup_asym, ut_teardown_asym, test_mod_inv),
+		TEST_CASE_ST(ut_setup_asym, ut_teardown_asym, test_mod_exp),
+		TEST_CASE_ST(ut_setup_asym, ut_teardown_asym, test_one_by_one),
 		TEST_CASES_END() /**< NULL terminate unit test array */
 	}
 };
@@ -2308,7 +2312,7 @@ static struct unit_test_suite cryptodev_qat_asym_testsuite  = {
 	.setup = testsuite_setup,
 	.teardown = testsuite_teardown,
 	.unit_test_cases = {
-		TEST_CASE_ST(ut_setup, ut_teardown, test_one_by_one),
+		TEST_CASE_ST(ut_setup_asym, ut_teardown_asym, test_one_by_one),
 		TEST_CASES_END() /**< NULL terminate unit test array */
 	}
 };
@@ -2318,13 +2322,16 @@ static struct unit_test_suite cryptodev_octeontx_asym_testsuite  = {
 	.setup = testsuite_setup,
 	.teardown = testsuite_teardown,
 	.unit_test_cases = {
-		TEST_CASE_ST(ut_setup, ut_teardown, test_capability),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_rsa_enc_dec_crt),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_rsa_sign_verify_crt),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_mod_exp),
-		TEST_CASE_ST(ut_setup, ut_teardown,
+		TEST_CASE_ST(ut_setup_asym, ut_teardown_asym, test_capability),
+		TEST_CASE_ST(ut_setup_asym, ut_teardown_asym,
+				test_rsa_enc_dec_crt),
+		TEST_CASE_ST(ut_setup_asym, ut_teardown_asym,
+				test_rsa_sign_verify_crt),
+		TEST_CASE_ST(ut_setup_asym, ut_teardown_asym, test_mod_exp),
+		TEST_CASE_ST(ut_setup_asym, ut_teardown_asym,
 			     test_ecdsa_sign_verify_all_curve),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_ecpm_all_curve),
+		TEST_CASE_ST(ut_setup_asym, ut_teardown_asym,
+				test_ecpm_all_curve),
 		TEST_CASES_END() /**< NULL terminate unit test array */
 	}
 };
diff --git a/app/test/test_ipsec.c b/app/test/test_ipsec.c
index d18220a885..fb90130ae2 100644
--- a/app/test/test_ipsec.c
+++ b/app/test/test_ipsec.c
@@ -424,7 +424,7 @@ testsuite_teardown(void)
 }
 
 static int
-ut_setup(void)
+ut_setup_ipsec(void)
 {
 	struct ipsec_testsuite_params *ts_params = &testsuite_params;
 	struct ipsec_unitest_params *ut_params = &unittest_params;
@@ -444,7 +444,7 @@ ut_setup(void)
 }
 
 static void
-ut_teardown(void)
+ut_teardown_ipsec(void)
 {
 	struct ipsec_testsuite_params *ts_params = &testsuite_params;
 	struct ipsec_unitest_params *ut_params = &unittest_params;
@@ -2499,33 +2499,33 @@ static struct unit_test_suite ipsec_testsuite  = {
 	.setup = testsuite_setup,
 	.teardown = testsuite_teardown,
 	.unit_test_cases = {
-		TEST_CASE_ST(ut_setup, ut_teardown,
+		TEST_CASE_ST(ut_setup_ipsec, ut_teardown_ipsec,
 			test_ipsec_crypto_inb_burst_null_null_wrapper),
-		TEST_CASE_ST(ut_setup, ut_teardown,
+		TEST_CASE_ST(ut_setup_ipsec, ut_teardown_ipsec,
 			test_ipsec_crypto_outb_burst_null_null_wrapper),
-		TEST_CASE_ST(ut_setup, ut_teardown,
+		TEST_CASE_ST(ut_setup_ipsec, ut_teardown_ipsec,
 			test_ipsec_inline_crypto_inb_burst_null_null_wrapper),
-		TEST_CASE_ST(ut_setup, ut_teardown,
+		TEST_CASE_ST(ut_setup_ipsec, ut_teardown_ipsec,
 			test_ipsec_inline_crypto_outb_burst_null_null_wrapper),
-		TEST_CASE_ST(ut_setup, ut_teardown,
+		TEST_CASE_ST(ut_setup_ipsec, ut_teardown_ipsec,
 			test_ipsec_inline_proto_inb_burst_null_null_wrapper),
-		TEST_CASE_ST(ut_setup, ut_teardown,
+		TEST_CASE_ST(ut_setup_ipsec, ut_teardown_ipsec,
 			test_ipsec_inline_proto_outb_burst_null_null_wrapper),
-		TEST_CASE_ST(ut_setup, ut_teardown,
+		TEST_CASE_ST(ut_setup_ipsec, ut_teardown_ipsec,
 			test_ipsec_lksd_proto_inb_burst_null_null_wrapper),
-		TEST_CASE_ST(ut_setup, ut_teardown,
+		TEST_CASE_ST(ut_setup_ipsec, ut_teardown_ipsec,
 			test_ipsec_lksd_proto_outb_burst_null_null_wrapper),
-		TEST_CASE_ST(ut_setup, ut_teardown,
+		TEST_CASE_ST(ut_setup_ipsec, ut_teardown_ipsec,
 			test_ipsec_replay_inb_inside_null_null_wrapper),
-		TEST_CASE_ST(ut_setup, ut_teardown,
+		TEST_CASE_ST(ut_setup_ipsec, ut_teardown_ipsec,
 			test_ipsec_replay_inb_outside_null_null_wrapper),
-		TEST_CASE_ST(ut_setup, ut_teardown,
+		TEST_CASE_ST(ut_setup_ipsec, ut_teardown_ipsec,
 			test_ipsec_replay_inb_repeat_null_null_wrapper),
-		TEST_CASE_ST(ut_setup, ut_teardown,
+		TEST_CASE_ST(ut_setup_ipsec, ut_teardown_ipsec,
 			test_ipsec_replay_inb_inside_burst_null_null_wrapper),
-		TEST_CASE_ST(ut_setup, ut_teardown,
+		TEST_CASE_ST(ut_setup_ipsec, ut_teardown_ipsec,
 			test_ipsec_crypto_inb_burst_2sa_null_null_wrapper),
-		TEST_CASE_ST(ut_setup, ut_teardown,
+		TEST_CASE_ST(ut_setup_ipsec, ut_teardown_ipsec,
 			test_ipsec_crypto_inb_burst_2sa_4grp_null_null_wrapper),
 		TEST_CASES_END() /**< NULL terminate unit test array */
 	}
-- 
2.25.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* [dpdk-dev] [PATCH v3 6/7] test/crypto: fix return value on test skipped
  2021-04-23 16:18 [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework Ciara Power
                   ` (4 preceding siblings ...)
  2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 5/7] test/crypto: move testsuite params to header file Ciara Power
@ 2021-04-23 16:18 ` Ciara Power
  2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 7/7] test/crypto: dynamically build blockcipher suite Ciara Power
                   ` (3 subsequent siblings)
  9 siblings, 0 replies; 15+ messages in thread
From: Ciara Power @ 2021-04-23 16:18 UTC (permalink / raw)
  To: dev
  Cc: thomas, declan.doherty, gakhil, aconole, hemant.agrawal, anoobj,
	ruifeng.wang, asomalap, ajit.khaparde, g.singh, Ciara Power,
	roy.fan.zhang, stable

The blockcipher testcase return value TEST_SUCCESS was incorrect for
one conditional check, it should have been TEST_SKIPPED similar to the
other condition checks in this function when the testcase is skipped.

Fixes: 4868f6591c6f ("test/crypto: add cases for raw datapath API")
Cc: roy.fan.zhang@intel.com
Cc: stable@dpdk.org

Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>
---
 app/test/test_cryptodev_blockcipher.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/app/test/test_cryptodev_blockcipher.c b/app/test/test_cryptodev_blockcipher.c
index b99d2ce50a..411968837f 100644
--- a/app/test/test_cryptodev_blockcipher.c
+++ b/app/test/test_cryptodev_blockcipher.c
@@ -170,7 +170,7 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t,
 			printf("Raw Data Path APIs do not support OOP, "
 				"Test Skipped.\n");
 			snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN, "SKIPPED");
-			status = TEST_SUCCESS;
+			status = TEST_SKIPPED;
 			goto error_exit;
 		}
 	}
-- 
2.25.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* [dpdk-dev] [PATCH v3 7/7] test/crypto: dynamically build blockcipher suite
  2021-04-23 16:18 [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework Ciara Power
                   ` (5 preceding siblings ...)
  2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 6/7] test/crypto: fix return value on test skipped Ciara Power
@ 2021-04-23 16:18 ` Ciara Power
  2021-04-25  9:24 ` [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework Ruifeng Wang
                   ` (2 subsequent siblings)
  9 siblings, 0 replies; 15+ messages in thread
From: Ciara Power @ 2021-04-23 16:18 UTC (permalink / raw)
  To: dev
  Cc: thomas, declan.doherty, gakhil, aconole, hemant.agrawal, anoobj,
	ruifeng.wang, asomalap, ajit.khaparde, g.singh, Ciara Power

In the existing implementation, the blockcipher test cases are being run
and reported as one test case per type, even though multiple test cases
are hidden in each. For example, "test_AES_chain_all" runs 46 test cases.
Each blockcipher type should have a testsuite instead.

The blockcipher testsuite is dynamically built, depending on the
blockcipher type chosen. The testcase struct is modified to allow
running a testcase with data, which is used for data required when
running each blockcipher testcase.

The blockcipher testsuites are added dynamically to parent testsuites
as sub-testsuites where needed.

Signed-off-by: Ciara Power <ciara.power@intel.com>

---
v3:
  - Modified release note to reflect allowing both nested testsuites
    and testcases.
  - Blockcipher testsuites now only need to be added to the cryptodev
    parent testsuite, and various scheduler sub-testsuites.
  - Setup functions are added for each blockcipher testsuite,
    and capabilities are checked in them.
v2:
  - Squashed release note patch into this patch.
  - Modified the build blockcipher suite function to use the testcases
    flexible array, and return a testsuite pointer.
---
 app/test/test.c                        |  11 +-
 app/test/test.h                        |  16 +-
 app/test/test_cryptodev.c              | 161 +++-------
 app/test/test_cryptodev_blockcipher.c  | 421 ++++++++++++++++++++++---
 app/test/test_cryptodev_blockcipher.h  |  12 +-
 doc/guides/rel_notes/release_21_05.rst |   5 +
 6 files changed, 447 insertions(+), 179 deletions(-)

diff --git a/app/test/test.c b/app/test/test.c
index ac0a66392a..173d202e47 100644
--- a/app/test/test.c
+++ b/app/test/test.c
@@ -38,7 +38,8 @@ extern cmdline_parse_ctx_t main_ctx[];
 
 #define FOR_EACH_SUITE_TESTCASE(iter, suite, case)			\
 	for (iter = 0, case = suite->unit_test_cases[0];		\
-		suite->unit_test_cases[iter].testcase;			\
+		suite->unit_test_cases[iter].testcase ||		\
+		suite->unit_test_cases[iter].testcase_with_data;	\
 		iter++, case = suite->unit_test_cases[iter])
 
 #define FOR_EACH_SUITE_TESTSUITE(iter, suite, sub_ts)			\
@@ -341,7 +342,13 @@ unit_test_suite_runner(struct unit_test_suite *suite)
 
 		if (test_success == TEST_SUCCESS) {
 			/* run the test case */
-			test_success = tc.testcase();
+			if (tc.testcase)
+				test_success = tc.testcase();
+			else if (tc.testcase_with_data)
+				test_success = tc.testcase_with_data(tc.data);
+			else
+				test_success = -ENOTSUP;
+
 			if (test_success == TEST_SUCCESS)
 				suite->succeeded++;
 			else if (test_success == TEST_SKIPPED)
diff --git a/app/test/test.h b/app/test/test.h
index f277df7c9d..c3b2a877ec 100644
--- a/app/test/test.h
+++ b/app/test/test.h
@@ -108,24 +108,28 @@ struct unit_test_case {
 	int (*setup)(void);
 	void (*teardown)(void);
 	int (*testcase)(void);
+	int (*testcase_with_data)(const void *data);
 	const char *name;
 	unsigned enabled;
+	const void *data;
 };
 
-#define TEST_CASE(fn) { NULL, NULL, fn, #fn, 1 }
+#define TEST_CASE(fn) { NULL, NULL, fn, NULL, #fn, 1, NULL }
 
-#define TEST_CASE_NAMED(name, fn) { NULL, NULL, fn, name, 1 }
+#define TEST_CASE_NAMED(name, fn) { NULL, NULL, fn, NULL, name, 1, NULL }
 
 #define TEST_CASE_ST(setup, teardown, testcase) \
-		{ setup, teardown, testcase, #testcase, 1 }
+		{ setup, teardown, testcase, NULL, #testcase, 1, NULL }
 
+#define TEST_CASE_WITH_DATA(setup, teardown, testcase, data) \
+		{ setup, teardown, NULL, testcase, #testcase, 1, data }
 
-#define TEST_CASE_DISABLED(fn) { NULL, NULL, fn, #fn, 0 }
+#define TEST_CASE_DISABLED(fn) { NULL, NULL, fn, NULL, #fn, 0, NULL }
 
 #define TEST_CASE_ST_DISABLED(setup, teardown, testcase) \
-		{ setup, teardown, testcase, #testcase, 0 }
+		{ setup, teardown, testcase, NULL, #testcase, 0, NULL }
 
-#define TEST_CASES_END() { NULL, NULL, NULL, NULL, 0 }
+#define TEST_CASES_END() { NULL, NULL, NULL, NULL, NULL, 0, NULL }
 
 static inline void
 debug_hexdump(FILE *file, const char *title, const void *buf, size_t len)
diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c
index 67adfedd72..f35171ad02 100644
--- a/app/test/test_cryptodev.c
+++ b/app/test/test_cryptodev.c
@@ -103,6 +103,15 @@ struct crypto_unittest_params {
 	for (j = 0; j < num_child_ts; index++, j++)			\
 		parent_ts.unit_test_suites[index] = child_ts[j]
 
+#define ADD_BLOCKCIPHER_TESTSUITE(index, parent_ts, blk_types, num_blk_types)	\
+	for (j = 0; j < num_blk_types; index++, j++)				\
+		parent_ts.unit_test_suites[index] =				\
+				build_blockcipher_test_suite(blk_types[j])
+
+#define FREE_BLOCKCIPHER_TESTSUITE(index, parent_ts, num_blk_types)		\
+	for (j = index; j < index + num_blk_types; j++)				\
+		free_blockcipher_test_suite(parent_ts.unit_test_suites[j])
+
 /*
  * Forward declarations.
  */
@@ -2312,80 +2321,6 @@ test_AES_CBC_HMAC_SHA512_decrypt_perform(struct rte_cryptodev_sym_session *sess,
 	return TEST_SUCCESS;
 }
 
-static int
-test_blockcipher(enum blockcipher_test_type test_type)
-{
-	struct crypto_testsuite_params *ts_params = &testsuite_params;
-	int status;
-
-	status = test_blockcipher_all_tests(ts_params->mbuf_pool,
-		ts_params->op_mpool,
-		ts_params->session_mpool, ts_params->session_priv_mpool,
-		ts_params->valid_devs[0],
-		test_type);
-
-	if (status == -ENOTSUP)
-		return status;
-
-	TEST_ASSERT_EQUAL(status, 0, "Test failed");
-
-	return TEST_SUCCESS;
-}
-
-static int
-test_AES_cipheronly_all(void)
-{
-	return test_blockcipher(BLKCIPHER_AES_CIPHERONLY_TYPE);
-}
-
-static int
-test_AES_docsis_all(void)
-{
-	/* Data-path service does not support DOCSIS yet */
-	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-		return -ENOTSUP;
-	return test_blockcipher(BLKCIPHER_AES_DOCSIS_TYPE);
-}
-
-static int
-test_DES_docsis_all(void)
-{
-	/* Data-path service does not support DOCSIS yet */
-	if (global_api_test_type == CRYPTODEV_RAW_API_TEST)
-		return -ENOTSUP;
-	return test_blockcipher(BLKCIPHER_DES_DOCSIS_TYPE);
-}
-
-static int
-test_DES_cipheronly_all(void)
-{
-	return test_blockcipher(BLKCIPHER_DES_CIPHERONLY_TYPE);
-}
-
-static int
-test_authonly_all(void)
-{
-	return test_blockcipher(BLKCIPHER_AUTHONLY_TYPE);
-}
-
-static int
-test_AES_chain_all(void)
-{
-	return test_blockcipher(BLKCIPHER_AES_CHAIN_TYPE);
-}
-
-static int
-test_3DES_chain_all(void)
-{
-	return test_blockcipher(BLKCIPHER_3DES_CHAIN_TYPE);
-}
-
-static int
-test_3DES_cipheronly_all(void)
-{
-	return test_blockcipher(BLKCIPHER_3DES_CIPHERONLY_TYPE);
-}
-
 /* ***** SNOW 3G Tests ***** */
 static int
 create_wireless_algo_hash_session(uint8_t dev_id,
@@ -13776,14 +13711,6 @@ static struct unit_test_suite cryptodev_gen_testsuite  = {
 				test_queue_pair_descriptor_setup),
 		TEST_CASE_ST(ut_setup, ut_teardown,
 				test_device_configure_invalid_queue_pair_ids),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_cipheronly_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_3DES_chain_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_3DES_cipheronly_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_DES_cipheronly_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_AES_docsis_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_DES_docsis_all),
-		TEST_CASE_ST(ut_setup, ut_teardown, test_authonly_all),
 		TEST_CASE_ST(ut_setup, ut_teardown, test_stats),
 		TEST_CASE_ST(ut_setup, ut_teardown, test_enq_callback_setup),
 		TEST_CASE_ST(ut_setup, ut_teardown, test_deq_callback_setup),
@@ -14511,7 +14438,16 @@ static struct unit_test_suite cryptodev_mixed_cipher_hash_testsuite  = {
 static int
 run_cryptodev_testsuite(const char *pmd_name)
 {
-	uint8_t ret, j, i = 0;
+	uint8_t ret, j, i = 0, blk_start_idx = 0;
+	const enum blockcipher_test_type blk_suites[] = {
+		BLKCIPHER_AES_CHAIN_TYPE,
+		BLKCIPHER_AES_CIPHERONLY_TYPE,
+		BLKCIPHER_AES_DOCSIS_TYPE,
+		BLKCIPHER_3DES_CHAIN_TYPE,
+		BLKCIPHER_3DES_CIPHERONLY_TYPE,
+		BLKCIPHER_DES_CIPHERONLY_TYPE,
+		BLKCIPHER_DES_DOCSIS_TYPE,
+		BLKCIPHER_AUTHONLY_TYPE};
 	struct unit_test_suite *static_suites[] = {
 		&cryptodev_multi_session_testsuite,
 		&cryptodev_null_testsuite,
@@ -14550,11 +14486,13 @@ run_cryptodev_testsuite(const char *pmd_name)
 	}
 
 	ts.unit_test_suites = malloc(sizeof(struct unit_test_suite *) *
-			RTE_DIM(static_suites));
+			(RTE_DIM(blk_suites) + RTE_DIM(static_suites)));
 
+	ADD_BLOCKCIPHER_TESTSUITE(i, ts, blk_suites, RTE_DIM(blk_suites));
 	ADD_STATIC_TESTSUITE(i, ts, static_suites, RTE_DIM(static_suites));
 	ret = unit_test_suite_runner(&ts);
 
+	FREE_BLOCKCIPHER_TESTSUITE(blk_start_idx, ts, RTE_DIM(blk_suites));
 	free(ts.unit_test_suites);
 	return ret;
 }
@@ -14652,54 +14590,35 @@ test_cryptodev_mrvl(void)
 static int
 test_cryptodev_scheduler(void /*argv __rte_unused, int argc __rte_unused*/)
 {
-	uint8_t ret, j, i = 0;
+	uint8_t ret, sched_i, j, i = 0, blk_start_idx = 0;
+	const enum blockcipher_test_type blk_suites[] = {
+		BLKCIPHER_AES_CHAIN_TYPE,
+		BLKCIPHER_AES_CIPHERONLY_TYPE,
+		BLKCIPHER_AUTHONLY_TYPE
+	};
 	static struct unit_test_suite scheduler_multicore = {
 		.suite_name = "Scheduler Multicore Unit Test Suite",
 		.setup = scheduler_multicore_testsuite_setup,
 		.teardown = scheduler_mode_testsuite_teardown,
-		.unit_test_cases = {
-			TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
-			TEST_CASE_ST(ut_setup, ut_teardown,
-					test_AES_cipheronly_all),
-			TEST_CASE_ST(ut_setup, ut_teardown, test_authonly_all),
-			TEST_CASES_END()
-		}
+		.unit_test_cases = {TEST_CASES_END()}
 	};
 	static struct unit_test_suite scheduler_round_robin = {
 		.suite_name = "Scheduler Round Robin Unit Test Suite",
 		.setup = scheduler_roundrobin_testsuite_setup,
 		.teardown = scheduler_mode_testsuite_teardown,
-		.unit_test_cases = {
-			TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
-			TEST_CASE_ST(ut_setup, ut_teardown,
-					test_AES_cipheronly_all),
-			TEST_CASE_ST(ut_setup, ut_teardown, test_authonly_all),
-			TEST_CASES_END()
-		}
+		.unit_test_cases = {TEST_CASES_END()}
 	};
 	static struct unit_test_suite scheduler_failover = {
 		.suite_name = "Scheduler Failover Unit Test Suite",
 		.setup = scheduler_failover_testsuite_setup,
 		.teardown = scheduler_mode_testsuite_teardown,
-		.unit_test_cases = {
-			TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
-			TEST_CASE_ST(ut_setup, ut_teardown,
-					test_AES_cipheronly_all),
-			TEST_CASE_ST(ut_setup, ut_teardown, test_authonly_all),
-			TEST_CASES_END()
-		}
+		.unit_test_cases = {TEST_CASES_END()}
 	};
 	static struct unit_test_suite scheduler_pkt_size_distr = {
 		.suite_name = "Scheduler Pkt Size Distr Unit Test Suite",
 		.setup = scheduler_pkt_size_distr_testsuite_setup,
 		.teardown = scheduler_mode_testsuite_teardown,
-		.unit_test_cases = {
-			TEST_CASE_ST(ut_setup, ut_teardown, test_AES_chain_all),
-			TEST_CASE_ST(ut_setup, ut_teardown,
-					test_AES_cipheronly_all),
-			TEST_CASE_ST(ut_setup, ut_teardown, test_authonly_all),
-			TEST_CASES_END()
-		}
+		.unit_test_cases = {TEST_CASES_END()}
 	};
 	struct unit_test_suite *sched_mode_suites[] = {
 		&scheduler_multicore,
@@ -14745,6 +14664,16 @@ test_cryptodev_scheduler(void /*argv __rte_unused, int argc __rte_unused*/)
 		return TEST_SKIPPED;
 	}
 
+	for (sched_i = 0; sched_i < RTE_DIM(sched_mode_suites); sched_i++) {
+		uint8_t blk_i = 0;
+		sched_mode_suites[sched_i]->unit_test_suites = malloc(sizeof
+				(struct unit_test_suite *) *
+				(RTE_DIM(blk_suites) + 1));
+		ADD_BLOCKCIPHER_TESTSUITE(blk_i, (*sched_mode_suites[sched_i]),
+				blk_suites, RTE_DIM(blk_suites));
+		sched_mode_suites[sched_i]->unit_test_suites[blk_i] = &end_testsuite;
+	}
+
 	ts.unit_test_suites = malloc(sizeof(struct unit_test_suite *) *
 			(RTE_DIM(static_suites) + RTE_DIM(sched_mode_suites)));
 	ADD_STATIC_TESTSUITE(i, ts, sched_mode_suites,
@@ -14752,6 +14681,12 @@ test_cryptodev_scheduler(void /*argv __rte_unused, int argc __rte_unused*/)
 	ADD_STATIC_TESTSUITE(i, ts, static_suites, RTE_DIM(static_suites));
 	ret = unit_test_suite_runner(&ts);
 
+	for (sched_i = 0; sched_i < RTE_DIM(sched_mode_suites); sched_i++) {
+		FREE_BLOCKCIPHER_TESTSUITE(blk_start_idx,
+				(*sched_mode_suites[sched_i]),
+				RTE_DIM(blk_suites));
+		free(sched_mode_suites[sched_i]->unit_test_suites);
+	}
 	free(ts.unit_test_suites);
 	return ret;
 }
diff --git a/app/test/test_cryptodev_blockcipher.c b/app/test/test_cryptodev_blockcipher.c
index 411968837f..d342c7b859 100644
--- a/app/test/test_cryptodev_blockcipher.c
+++ b/app/test/test_cryptodev_blockcipher.c
@@ -813,82 +813,401 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t,
 	return status;
 }
 
-int
-test_blockcipher_all_tests(struct rte_mempool *mbuf_pool,
-	struct rte_mempool *op_mpool,
-	struct rte_mempool *sess_mpool,
-	struct rte_mempool *sess_priv_mpool,
-	uint8_t dev_id,
-	enum blockcipher_test_type test_type)
+static int
+blockcipher_test_case_run(const void *data)
 {
-	int status, overall_status = TEST_SUCCESS;
-	uint32_t i, test_index = 0;
+	const struct blockcipher_test_case *tc_data = data;
+	int status;
 	char test_msg[BLOCKCIPHER_TEST_MSG_LEN + 1];
-	uint32_t n_test_cases = 0;
-	const struct blockcipher_test_case *tcs = NULL;
+
+	status = test_blockcipher_one_case(tc_data,
+			p_testsuite_params->mbuf_pool,
+			p_testsuite_params->op_mpool,
+			p_testsuite_params->session_mpool,
+			p_testsuite_params->session_priv_mpool,
+			p_testsuite_params->valid_devs[0],
+			test_msg);
+	return status;
+}
+
+static int
+aes_chain_setup(void)
+{
+	uint8_t dev_id = p_testsuite_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	uint64_t feat_flags;
+	const enum rte_crypto_cipher_algorithm ciphers[] = {
+		RTE_CRYPTO_CIPHER_NULL,
+		RTE_CRYPTO_CIPHER_AES_CTR,
+		RTE_CRYPTO_CIPHER_AES_CBC
+	};
+	const enum rte_crypto_auth_algorithm auths[] = {
+		RTE_CRYPTO_AUTH_NULL,
+		RTE_CRYPTO_AUTH_SHA1_HMAC,
+		RTE_CRYPTO_AUTH_AES_XCBC_MAC,
+		RTE_CRYPTO_AUTH_SHA256_HMAC,
+		RTE_CRYPTO_AUTH_SHA512_HMAC,
+		RTE_CRYPTO_AUTH_SHA224_HMAC,
+		RTE_CRYPTO_AUTH_SHA384_HMAC
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+	feat_flags = dev_info.feature_flags;
+
+	if (!(feat_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for AES Chain "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_cipher_capabilities_supported(ciphers, RTE_DIM(ciphers)) != 0
+			&& check_auth_capabilities_supported(auths,
+			RTE_DIM(auths)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for AES Chain "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+aes_cipheronly_setup(void)
+{
+	uint8_t dev_id = p_testsuite_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	uint64_t feat_flags;
+	const enum rte_crypto_cipher_algorithm ciphers[] = {
+		RTE_CRYPTO_CIPHER_NULL,
+		RTE_CRYPTO_CIPHER_AES_CTR,
+		RTE_CRYPTO_CIPHER_AES_CBC,
+		RTE_CRYPTO_CIPHER_AES_ECB,
+		RTE_CRYPTO_CIPHER_AES_XTS
+	};
+	const enum rte_crypto_auth_algorithm auths[] = {
+		RTE_CRYPTO_AUTH_NULL,
+		RTE_CRYPTO_AUTH_SHA1_HMAC,
+		RTE_CRYPTO_AUTH_AES_XCBC_MAC
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+	feat_flags = dev_info.feature_flags;
+
+	if (!(feat_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for AES Cipheronly "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_cipher_capabilities_supported(ciphers, RTE_DIM(ciphers)) != 0
+			&& check_auth_capabilities_supported(auths,
+			RTE_DIM(auths)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for AES Cipheronly "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+aes_docsis_setup(void)
+{
+	uint8_t dev_id = p_testsuite_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	uint64_t feat_flags;
+	const enum rte_crypto_cipher_algorithm ciphers[] = {
+		RTE_CRYPTO_CIPHER_AES_DOCSISBPI
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+	feat_flags = dev_info.feature_flags;
+
+	/* Data-path service does not support DOCSIS yet */
+	if (!(feat_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			(global_api_test_type == CRYPTODEV_RAW_API_TEST)) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for AES Docsis "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_cipher_capabilities_supported(ciphers, RTE_DIM(ciphers)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for AES Docsis "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+triple_des_chain_setup(void)
+{
+	uint8_t dev_id = p_testsuite_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	uint64_t feat_flags;
+	const enum rte_crypto_cipher_algorithm ciphers[] = {
+		RTE_CRYPTO_CIPHER_3DES_CTR,
+		RTE_CRYPTO_CIPHER_3DES_CBC
+	};
+	const enum rte_crypto_auth_algorithm auths[] = {
+		RTE_CRYPTO_AUTH_SHA1_HMAC,
+		RTE_CRYPTO_AUTH_SHA1
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+	feat_flags = dev_info.feature_flags;
+
+	if (!(feat_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for 3DES Chain "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_cipher_capabilities_supported(ciphers, RTE_DIM(ciphers)) != 0
+			&& check_auth_capabilities_supported(auths,
+			RTE_DIM(auths)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for 3DES Chain "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+triple_des_cipheronly_setup(void)
+{
+	uint8_t dev_id = p_testsuite_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	uint64_t feat_flags;
+	const enum rte_crypto_cipher_algorithm ciphers[] = {
+		RTE_CRYPTO_CIPHER_3DES_CTR,
+		RTE_CRYPTO_CIPHER_3DES_CBC
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+	feat_flags = dev_info.feature_flags;
+
+	if (!(feat_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for 3DES "
+				"Cipheronly testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_cipher_capabilities_supported(ciphers, RTE_DIM(ciphers)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for 3DES "
+				"Cipheronly testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+des_cipheronly_setup(void)
+{
+	uint8_t dev_id = p_testsuite_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	uint64_t feat_flags;
+	const enum rte_crypto_cipher_algorithm ciphers[] = {
+		RTE_CRYPTO_CIPHER_DES_CBC
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+	feat_flags = dev_info.feature_flags;
+
+	if (!(feat_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for DES "
+				"Cipheronly testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_cipher_capabilities_supported(ciphers, RTE_DIM(ciphers)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for DES "
+				"Cipheronly testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+des_docsis_setup(void)
+{
+	uint8_t dev_id = p_testsuite_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	uint64_t feat_flags;
+	const enum rte_crypto_cipher_algorithm ciphers[] = {
+		RTE_CRYPTO_CIPHER_DES_DOCSISBPI
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+	feat_flags = dev_info.feature_flags;
+
+	/* Data-path service does not support DOCSIS yet */
+	if (!(feat_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			(global_api_test_type == CRYPTODEV_RAW_API_TEST)) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for DES Docsis "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_cipher_capabilities_supported(ciphers, RTE_DIM(ciphers)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for DES Docsis "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+static int
+authonly_setup(void)
+{
+	uint8_t dev_id = p_testsuite_params->valid_devs[0];
+	struct rte_cryptodev_info dev_info;
+	uint64_t feat_flags;
+	const enum rte_crypto_auth_algorithm auths[] = {
+		RTE_CRYPTO_AUTH_MD5,
+		RTE_CRYPTO_AUTH_MD5_HMAC,
+		RTE_CRYPTO_AUTH_SHA1,
+		RTE_CRYPTO_AUTH_SHA1_HMAC,
+		RTE_CRYPTO_AUTH_SHA224,
+		RTE_CRYPTO_AUTH_SHA224_HMAC,
+		RTE_CRYPTO_AUTH_SHA256,
+		RTE_CRYPTO_AUTH_SHA256_HMAC,
+		RTE_CRYPTO_AUTH_SHA384,
+		RTE_CRYPTO_AUTH_SHA384_HMAC,
+		RTE_CRYPTO_AUTH_SHA512,
+		RTE_CRYPTO_AUTH_SHA512_HMAC,
+		RTE_CRYPTO_AUTH_AES_CMAC,
+		RTE_CRYPTO_AUTH_NULL,
+		RTE_CRYPTO_AUTH_AES_XCBC_MAC
+	};
+
+	rte_cryptodev_info_get(dev_id, &dev_info);
+	feat_flags = dev_info.feature_flags;
+
+	if (!(feat_flags & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) ||
+			((global_api_test_type == CRYPTODEV_RAW_API_TEST) &&
+			!(feat_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP))) {
+		RTE_LOG(INFO, USER1, "Feature flag requirements for Auth Only "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	if (check_auth_capabilities_supported(auths, RTE_DIM(auths)) != 0) {
+		RTE_LOG(INFO, USER1, "Capability requirements for Auth Only "
+				"testsuite not met\n");
+		return TEST_SKIPPED;
+	}
+
+	return 0;
+}
+
+struct unit_test_suite *
+build_blockcipher_test_suite(enum blockcipher_test_type test_type)
+{
+	int i, n_test_cases = 0;
+	struct unit_test_suite *ts;
+	const char *ts_name = NULL;
+	const struct blockcipher_test_case *blk_tcs;
+	struct unit_test_case *tc;
+	int (*ts_setup)(void) = NULL;
 
 	switch (test_type) {
 	case BLKCIPHER_AES_CHAIN_TYPE:
-		n_test_cases = sizeof(aes_chain_test_cases) /
-		sizeof(aes_chain_test_cases[0]);
-		tcs = aes_chain_test_cases;
+		n_test_cases = RTE_DIM(aes_chain_test_cases);
+		blk_tcs = aes_chain_test_cases;
+		ts_name = "AES Chain";
+		ts_setup = aes_chain_setup;
 		break;
 	case BLKCIPHER_AES_CIPHERONLY_TYPE:
-		n_test_cases = sizeof(aes_cipheronly_test_cases) /
-		sizeof(aes_cipheronly_test_cases[0]);
-		tcs = aes_cipheronly_test_cases;
+		n_test_cases = RTE_DIM(aes_cipheronly_test_cases);
+		blk_tcs = aes_cipheronly_test_cases;
+		ts_name = "AES Cipher Only";
+		ts_setup = aes_cipheronly_setup;
 		break;
 	case BLKCIPHER_AES_DOCSIS_TYPE:
-		n_test_cases = sizeof(aes_docsis_test_cases) /
-		sizeof(aes_docsis_test_cases[0]);
-		tcs = aes_docsis_test_cases;
+		n_test_cases = RTE_DIM(aes_docsis_test_cases);
+		blk_tcs = aes_docsis_test_cases;
+		ts_name = "AES Docsis";
+		ts_setup = aes_docsis_setup;
 		break;
 	case BLKCIPHER_3DES_CHAIN_TYPE:
-		n_test_cases = sizeof(triple_des_chain_test_cases) /
-		sizeof(triple_des_chain_test_cases[0]);
-		tcs = triple_des_chain_test_cases;
+		n_test_cases = RTE_DIM(triple_des_chain_test_cases);
+		blk_tcs = triple_des_chain_test_cases;
+		ts_name = "3DES Chain";
+		ts_setup = triple_des_chain_setup;
 		break;
 	case BLKCIPHER_3DES_CIPHERONLY_TYPE:
-		n_test_cases = sizeof(triple_des_cipheronly_test_cases) /
-		sizeof(triple_des_cipheronly_test_cases[0]);
-		tcs = triple_des_cipheronly_test_cases;
+		n_test_cases = RTE_DIM(triple_des_cipheronly_test_cases);
+		blk_tcs = triple_des_cipheronly_test_cases;
+		ts_name = "3DES Cipher Only";
+		ts_setup = triple_des_cipheronly_setup;
 		break;
 	case BLKCIPHER_DES_CIPHERONLY_TYPE:
-		n_test_cases = sizeof(des_cipheronly_test_cases) /
-		sizeof(des_cipheronly_test_cases[0]);
-		tcs = des_cipheronly_test_cases;
+		n_test_cases = RTE_DIM(des_cipheronly_test_cases);
+		blk_tcs = des_cipheronly_test_cases;
+		ts_name = "DES Cipher Only";
+		ts_setup = des_cipheronly_setup;
 		break;
 	case BLKCIPHER_DES_DOCSIS_TYPE:
-		n_test_cases = sizeof(des_docsis_test_cases) /
-		sizeof(des_docsis_test_cases[0]);
-		tcs = des_docsis_test_cases;
+		n_test_cases = RTE_DIM(des_docsis_test_cases);
+		blk_tcs = des_docsis_test_cases;
+		ts_name = "DES Docsis";
+		ts_setup = des_docsis_setup;
 		break;
 	case BLKCIPHER_AUTHONLY_TYPE:
-		n_test_cases = sizeof(hash_test_cases) /
-		sizeof(hash_test_cases[0]);
-		tcs = hash_test_cases;
+		n_test_cases = RTE_DIM(hash_test_cases);
+		blk_tcs = hash_test_cases;
+		ts_name = "Auth Only";
+		ts_setup = authonly_setup;
 		break;
 	default:
 		break;
 	}
 
-	for (i = 0; i < n_test_cases; i++) {
-		const struct blockcipher_test_case *tc = &tcs[i];
+	ts = calloc(1, sizeof(struct unit_test_suite) +
+			(sizeof(struct unit_test_case) * (n_test_cases + 1)));
+	ts->suite_name = ts_name;
+	ts->setup = ts_setup;
 
-		status = test_blockcipher_one_case(tc, mbuf_pool, op_mpool,
-			sess_mpool, sess_priv_mpool, dev_id,
-			test_msg);
-
-		printf("  %u) TestCase %s %s\n", test_index ++,
-			tc->test_descr, test_msg);
-
-		if (status == TEST_FAILED) {
-			overall_status = status;
-
-			if (tc->feature_mask & BLOCKCIPHER_TEST_FEATURE_STOPPER)
-				break;
-		}
+	for (i = 0; i < n_test_cases; i++) {
+		tc = &ts->unit_test_cases[i];
+		tc->name = blk_tcs[i].test_descr;
+		tc->enabled = 1;
+		tc->setup = ut_setup;
+		tc->teardown = ut_teardown;
+		tc->testcase = NULL;
+		tc->testcase_with_data = blockcipher_test_case_run;
+		tc->data = &blk_tcs[i];
 	}
+	tc = &ts->unit_test_cases[i];
+	tc->name = NULL;
+	tc->enabled = 0;
+	tc->setup = NULL;
+	tc->teardown = NULL;
+	tc->testcase = NULL;
+	tc->testcase_with_data = NULL;
+	tc->data = NULL;
+
+	return ts;
+}
 
-	return overall_status;
+void
+free_blockcipher_test_suite(struct unit_test_suite *ts)
+{
+	free(ts);
 }
diff --git a/app/test/test_cryptodev_blockcipher.h b/app/test/test_cryptodev_blockcipher.h
index 145d8da07e..a06241b06d 100644
--- a/app/test/test_cryptodev_blockcipher.h
+++ b/app/test/test_cryptodev_blockcipher.h
@@ -99,12 +99,10 @@ struct blockcipher_test_data {
 	unsigned int auth_offset;
 };
 
-int
-test_blockcipher_all_tests(struct rte_mempool *mbuf_pool,
-	struct rte_mempool *op_mpool,
-	struct rte_mempool *sess_mpool,
-	struct rte_mempool *sess_priv_mpool,
-	uint8_t dev_id,
-	enum blockcipher_test_type test_type);
+struct unit_test_suite *
+build_blockcipher_test_suite(enum blockcipher_test_type test_type);
+
+void
+free_blockcipher_test_suite(struct unit_test_suite *ts);
 
 #endif /* TEST_CRYPTODEV_BLOCKCIPHER_H_ */
diff --git a/doc/guides/rel_notes/release_21_05.rst b/doc/guides/rel_notes/release_21_05.rst
index b3224dc332..9ab6a36422 100644
--- a/doc/guides/rel_notes/release_21_05.rst
+++ b/doc/guides/rel_notes/release_21_05.rst
@@ -271,6 +271,11 @@ New Features
   * Added support for crypto adapter forward mode in octeontx2 event and crypto
     device driver.
 
+* **Added sub-testsuite support.**
+
+  * The unit test suite struct now supports having both a nested
+    list of sub-testsuites, and a list of testcases as before.
+
 
 Removed Items
 -------------
-- 
2.25.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework
  2021-04-23 16:18 [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework Ciara Power
                   ` (6 preceding siblings ...)
  2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 7/7] test/crypto: dynamically build blockcipher suite Ciara Power
@ 2021-04-25  9:24 ` Ruifeng Wang
  2021-04-30 21:00 ` Doherty, Declan
  2021-05-08 15:38 ` [dpdk-dev] [EXT] " Akhil Goyal
  9 siblings, 0 replies; 15+ messages in thread
From: Ruifeng Wang @ 2021-04-25  9:24 UTC (permalink / raw)
  To: Ciara Power, dev
  Cc: thomas, declan.doherty, gakhil, aconole, hemant.agrawal, anoobj,
	asomalap, Ajit Khaparde (ajit.khaparde@broadcom.com),
	g.singh, nd

> -----Original Message-----
> From: Ciara Power <ciara.power@intel.com>
> Sent: Saturday, April 24, 2021 12:18 AM
> To: dev@dpdk.org
> Cc: thomas@monjalon.net; declan.doherty@intel.com; gakhil@marvell.com;
> aconole@redhat.com; hemant.agrawal@nxp.com; anoobj@marvell.com;
> Ruifeng Wang <Ruifeng.Wang@arm.com>; asomalap@amd.com; Ajit
> Khaparde (ajit.khaparde@broadcom.com) <ajit.khaparde@broadcom.com>;
> g.singh@nxp.com; Ciara Power <ciara.power@intel.com>
> Subject: [PATCH v3 0/7] test: refactor crypto unit test framework
> 
> The current crypto unit test framework is not granular enough to accurately
> track unit test results. This is caused by one testcase in a suite actually
> running multiple testcases, but only returning one result.
> 
> The approach taken in this patchset allows a test suite have a list of sub-
> testsuites, and/or a list of testcases as previously used.
> The unit test suite runner can then recursively iterate and run the sub-
> testsuites, until it reaches a suite with testcases, and it then runs each
> testcase as it had done previously.
> In the case of a testsuite with both testcases and sub-testsuites, the
> testcases are executed first before iterating through the sub-testsuites.
> 
> By allowing this further breakdown into sub-testsuites, a refactor of the
> crypto unit tests solves the issue of inaccurate reporting, as sub-testsuites
> can be used in place of the testcases that had multiple testcases hidden on a
> sub level.
> The blockcipher tests previously had these hidden testcases, but are now
> sub-testsuites that are dynamically created and added to a parent test suite,
> allowing for each testcase status to be reported directly to the runner.
> The cryptodev test suite is broken down into smaller suites that are used as
> sub-testsuites, which allows for more flexibility choosing which sub-
> testsuites should run for the current device.
> The introduction of sub-testsuites also allows for more precise
> setup/teardown functions, that can check the capabilities required to run its
> testcases.
> 
> For example, when running the cryptodev_aesni_mb_autotest, the parent
> Cryptodev Test Suite is executed.
> Various sub-testsuites are added to the parent test suite, such as the static
> suites of testcases that were once in the cryptodev_testsuite, and
> blockcipher suites.
> The unit test runner can then run the Cryptodev parent test suite, which in
> turn will run the sub-testsuites.
> 
> The user is now required to create vdevs via EAL commandline args, this is no
> longer done within the test app for crypto autotests.
> 
> Documentation will need to be added at a later stage, adding to the test
> document that isn't yet merged. [1]
> 
> [1]
> https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536-1-
> aconole@redhat.com/
> 
> 
> v3:
>   - Added support for a testsuite having both a list of testcases,
>     and a list of sub-testsuites.
>   - Replaced PMD based parent testsuites with a cryptodev testsuite
>     used by all autotests, with the exception of scheduler autotest.
>   - Setup functions were added for all sub-testsuites, within which
>     required capability support is checked.
>   - The setup functions no longer create vdevs if needed,
>     this must be done by the user when running the test.
>   - Patch added to standardise return values for skipped testcases.
> v2:
>   - Added macro in place of testcase/testsuite loops.
>   - Added more detail in the summary output.
>   - Moved testcase counts to the testsuite structure.
>   - Flattened testsuite structure to remove union.
>   - Added patch for fix of blockcipher test return value.
>   - Squashed release note into last patch.
> 
> 
> Ciara Power (7):
>   app/test: refactor of unit test suite runner
>   test: introduce parent testsuite format
>   test/crypto: refactor to use sub-testsuites
>   test/crypto: replace unsupported with skipped
>   test/crypto: move testsuite params to header file
>   test/crypto: fix return value on test skipped
>   test/crypto: dynamically build blockcipher suite
> 
>  app/test/test.c                        |  176 +-
>  app/test/test.h                        |   23 +-
>  app/test/test_cryptodev.c              | 2324 ++++++++++++++----------
>  app/test/test_cryptodev.h              |   32 +
>  app/test/test_cryptodev_asym.c         |  111 +-
>  app/test/test_cryptodev_blockcipher.c  |  423 ++++-
>  app/test/test_cryptodev_blockcipher.h  |   12 +-
>  app/test/test_ipsec.c                  |   32 +-
>  doc/guides/rel_notes/release_21_05.rst |    5 +
>  9 files changed, 2000 insertions(+), 1138 deletions(-)
> 
> --
> 2.25.1

Tested armv8 cryptodev, and result looks good.
Tested-by: Ruifeng Wang <ruifeng.wang@arm.com>

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework
  2021-04-23 16:18 [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework Ciara Power
                   ` (7 preceding siblings ...)
  2021-04-25  9:24 ` [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework Ruifeng Wang
@ 2021-04-30 21:00 ` Doherty, Declan
  2021-05-05 10:41   ` Hemant Agrawal
  2021-05-08 15:38 ` [dpdk-dev] [EXT] " Akhil Goyal
  9 siblings, 1 reply; 15+ messages in thread
From: Doherty, Declan @ 2021-04-30 21:00 UTC (permalink / raw)
  To: Ciara Power, dev
  Cc: thomas, gakhil, aconole, hemant.agrawal, anoobj, ruifeng.wang,
	asomalap, ajit.khaparde, g.singh



On 23/04/2021 5:18 PM, Ciara Power wrote:
> The current crypto unit test framework is not granular enough to
> accurately track unit test results. This is caused by one testcase
> in a suite actually running multiple testcases, but only returning
> one result.
>   
> The approach taken in this patchset allows a test suite have a
> list of sub-testsuites, and/or a list of testcases as previously used.
> The unit test suite runner can then recursively iterate and run the
> sub-testsuites, until it reaches a suite with testcases,
> and it then runs each testcase as it had done previously.
> In the case of a testsuite with both testcases and sub-testsuites,
> the testcases are executed first before iterating through the
> sub-testsuites.
>   
> By allowing this further breakdown into sub-testsuites,
> a refactor of the crypto unit tests solves the issue of inaccurate
> reporting, as sub-testsuites can be used in place of the testcases
> that had multiple testcases hidden on a sub level.
> The blockcipher tests previously had these hidden testcases,
> but are now sub-testsuites that are dynamically created and added to a
> parent test suite, allowing for each testcase status to be reported
> directly to the runner.
> The cryptodev test suite is broken down into smaller suites that are
> used as sub-testsuites, which allows for more flexibility choosing which
> sub-testsuites should run for the current device.
> The introduction of sub-testsuites also allows for more precise
> setup/teardown functions, that can check the capabilities required to
> run its testcases.
>   
> For example, when running the cryptodev_aesni_mb_autotest,
> the parent Cryptodev Test Suite is executed.
> Various sub-testsuites are added to the parent test suite, such as
> the static suites of testcases that were once in the cryptodev_testsuite,
> and blockcipher suites.
> The unit test runner can then run the Cryptodev parent test suite,
> which in turn will run the sub-testsuites.
> 
> The user is now required to create vdevs via EAL commandline args,
> this is no longer done within the test app for crypto autotests.
> 
> Documentation will need to be added at a later stage,
> adding to the test document that isn't yet merged. [1]
> 
> [1] https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536-1-aconole@redhat.com/
> 
> 
> v3:
>    - Added support for a testsuite having both a list of testcases,
>      and a list of sub-testsuites.
>    - Replaced PMD based parent testsuites with a cryptodev testsuite
>      used by all autotests, with the exception of scheduler autotest.
>    - Setup functions were added for all sub-testsuites, within which
>      required capability support is checked.
>    - The setup functions no longer create vdevs if needed,
>      this must be done by the user when running the test.
>    - Patch added to standardise return values for skipped testcases.
> v2:
>    - Added macro in place of testcase/testsuite loops.
>    - Added more detail in the summary output.
>    - Moved testcase counts to the testsuite structure.
>    - Flattened testsuite structure to remove union.
>    - Added patch for fix of blockcipher test return value.
>    - Squashed release note into last patch.
> 
> 
> Ciara Power (7):
>    app/test: refactor of unit test suite runner
>    test: introduce parent testsuite format
>    test/crypto: refactor to use sub-testsuites
>    test/crypto: replace unsupported with skipped
>    test/crypto: move testsuite params to header file
>    test/crypto: fix return value on test skipped
>    test/crypto: dynamically build blockcipher suite
> 
>   app/test/test.c                        |  176 +-
>   app/test/test.h                        |   23 +-
>   app/test/test_cryptodev.c              | 2324 ++++++++++++++----------
>   app/test/test_cryptodev.h              |   32 +
>   app/test/test_cryptodev_asym.c         |  111 +-
>   app/test/test_cryptodev_blockcipher.c  |  423 ++++-
>   app/test/test_cryptodev_blockcipher.h  |   12 +-
>   app/test/test_ipsec.c                  |   32 +-
>   doc/guides/rel_notes/release_21_05.rst |    5 +
>   9 files changed, 2000 insertions(+), 1138 deletions(-)
> 

Series Acked-by: Declan Doherty <declan.doherty@intel.com>

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework
  2021-04-30 21:00 ` Doherty, Declan
@ 2021-05-05 10:41   ` Hemant Agrawal
  2021-05-05 12:20     ` Hemant Agrawal
  0 siblings, 1 reply; 15+ messages in thread
From: Hemant Agrawal @ 2021-05-05 10:41 UTC (permalink / raw)
  To: Doherty, Declan, Ciara Power, dev
  Cc: thomas, gakhil, aconole, hemant.agrawal, anoobj, ruifeng.wang,
	asomalap, ajit.khaparde, g.singh

Sorry for top posting.

Akhil,

     Will you please hold for a day on this patch? we are seeing few 
failures on dpaa2 with this patch, currently debugging them.


Regards,

Hemant

On 5/1/2021 2:30 AM, Doherty, Declan wrote:
>
>
> On 23/04/2021 5:18 PM, Ciara Power wrote:
>> The current crypto unit test framework is not granular enough to
>> accurately track unit test results. This is caused by one testcase
>> in a suite actually running multiple testcases, but only returning
>> one result.
>>   The approach taken in this patchset allows a test suite have a
>> list of sub-testsuites, and/or a list of testcases as previously used.
>> The unit test suite runner can then recursively iterate and run the
>> sub-testsuites, until it reaches a suite with testcases,
>> and it then runs each testcase as it had done previously.
>> In the case of a testsuite with both testcases and sub-testsuites,
>> the testcases are executed first before iterating through the
>> sub-testsuites.
>>   By allowing this further breakdown into sub-testsuites,
>> a refactor of the crypto unit tests solves the issue of inaccurate
>> reporting, as sub-testsuites can be used in place of the testcases
>> that had multiple testcases hidden on a sub level.
>> The blockcipher tests previously had these hidden testcases,
>> but are now sub-testsuites that are dynamically created and added to a
>> parent test suite, allowing for each testcase status to be reported
>> directly to the runner.
>> The cryptodev test suite is broken down into smaller suites that are
>> used as sub-testsuites, which allows for more flexibility choosing which
>> sub-testsuites should run for the current device.
>> The introduction of sub-testsuites also allows for more precise
>> setup/teardown functions, that can check the capabilities required to
>> run its testcases.
>>   For example, when running the cryptodev_aesni_mb_autotest,
>> the parent Cryptodev Test Suite is executed.
>> Various sub-testsuites are added to the parent test suite, such as
>> the static suites of testcases that were once in the 
>> cryptodev_testsuite,
>> and blockcipher suites.
>> The unit test runner can then run the Cryptodev parent test suite,
>> which in turn will run the sub-testsuites.
>>
>> The user is now required to create vdevs via EAL commandline args,
>> this is no longer done within the test app for crypto autotests.
>>
>> Documentation will need to be added at a later stage,
>> adding to the test document that isn't yet merged. [1]
>>
>> [1] 
>> https://patchwork.dpdk.org/project/dpdk/patch/20210309155757.615536-1-aconole@redhat.com/
>>
>>
>> v3:
>>    - Added support for a testsuite having both a list of testcases,
>>      and a list of sub-testsuites.
>>    - Replaced PMD based parent testsuites with a cryptodev testsuite
>>      used by all autotests, with the exception of scheduler autotest.
>>    - Setup functions were added for all sub-testsuites, within which
>>      required capability support is checked.
>>    - The setup functions no longer create vdevs if needed,
>>      this must be done by the user when running the test.
>>    - Patch added to standardise return values for skipped testcases.
>> v2:
>>    - Added macro in place of testcase/testsuite loops.
>>    - Added more detail in the summary output.
>>    - Moved testcase counts to the testsuite structure.
>>    - Flattened testsuite structure to remove union.
>>    - Added patch for fix of blockcipher test return value.
>>    - Squashed release note into last patch.
>>
>>
>> Ciara Power (7):
>>    app/test: refactor of unit test suite runner
>>    test: introduce parent testsuite format
>>    test/crypto: refactor to use sub-testsuites
>>    test/crypto: replace unsupported with skipped
>>    test/crypto: move testsuite params to header file
>>    test/crypto: fix return value on test skipped
>>    test/crypto: dynamically build blockcipher suite
>>
>>   app/test/test.c                        |  176 +-
>>   app/test/test.h                        |   23 +-
>>   app/test/test_cryptodev.c              | 2324 ++++++++++++++----------
>>   app/test/test_cryptodev.h              |   32 +
>>   app/test/test_cryptodev_asym.c         |  111 +-
>>   app/test/test_cryptodev_blockcipher.c  |  423 ++++-
>>   app/test/test_cryptodev_blockcipher.h  |   12 +-
>>   app/test/test_ipsec.c                  |   32 +-
>>   doc/guides/rel_notes/release_21_05.rst |    5 +
>>   9 files changed, 2000 insertions(+), 1138 deletions(-)
>>
>
> Series Acked-by: Declan Doherty <declan.doherty@intel.com>

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework
  2021-05-05 10:41   ` Hemant Agrawal
@ 2021-05-05 12:20     ` Hemant Agrawal
  0 siblings, 0 replies; 15+ messages in thread
From: Hemant Agrawal @ 2021-05-05 12:20 UTC (permalink / raw)
  To: Doherty, Declan, Ciara Power, dev
  Cc: thomas, gakhil, aconole, hemant.agrawal, anoobj, ruifeng.wang,
	asomalap, ajit.khaparde, g.singh

The issue was in dpaa2_sec driver, fix is submitted.

Series Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>


On 5/5/2021 4:11 PM, Hemant Agrawal wrote:
> Sorry for top posting.
>
> Akhil,
>
>     Will you please hold for a day on this patch? we are seeing few 
> failures on dpaa2 with this patch, currently debugging them.
>
>
> Regards,
>
> Hemant
>

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [dpdk-dev] [EXT] [PATCH v3 0/7] test: refactor crypto unit test framework
  2021-04-23 16:18 [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework Ciara Power
                   ` (8 preceding siblings ...)
  2021-04-30 21:00 ` Doherty, Declan
@ 2021-05-08 15:38 ` Akhil Goyal
  9 siblings, 0 replies; 15+ messages in thread
From: Akhil Goyal @ 2021-05-08 15:38 UTC (permalink / raw)
  To: Ciara Power, dev
  Cc: thomas, declan.doherty, aconole, hemant.agrawal, Anoob Joseph,
	ruifeng.wang, asomalap, ajit.khaparde, g.singh

Hi Ciara,

I see that case " test_snow3g_auth_cipher_test_case_1 " was unsupported
on marvell PMDs  and was getting skipped, but now after your series
it is causing segmentation fault. I guess there is some check missed for
digest encrypted case.

Can you check?

> The current crypto unit test framework is not granular enough to
> accurately track unit test results. This is caused by one testcase
> in a suite actually running multiple testcases, but only returning
> one result.
> 
> The approach taken in this patchset allows a test suite have a
> list of sub-testsuites, and/or a list of testcases as previously used.
> The unit test suite runner can then recursively iterate and run the
> sub-testsuites, until it reaches a suite with testcases,
> and it then runs each testcase as it had done previously.
> In the case of a testsuite with both testcases and sub-testsuites,
> the testcases are executed first before iterating through the
> sub-testsuites.
> 
> By allowing this further breakdown into sub-testsuites,
> a refactor of the crypto unit tests solves the issue of inaccurate
> reporting, as sub-testsuites can be used in place of the testcases
> that had multiple testcases hidden on a sub level.
> The blockcipher tests previously had these hidden testcases,
> but are now sub-testsuites that are dynamically created and added to a
> parent test suite, allowing for each testcase status to be reported
> directly to the runner.
> The cryptodev test suite is broken down into smaller suites that are
> used as sub-testsuites, which allows for more flexibility choosing which
> sub-testsuites should run for the current device.
> The introduction of sub-testsuites also allows for more precise
> setup/teardown functions, that can check the capabilities required to
> run its testcases.
> 
> For example, when running the cryptodev_aesni_mb_autotest,
> the parent Cryptodev Test Suite is executed.
> Various sub-testsuites are added to the parent test suite, such as
> the static suites of testcases that were once in the cryptodev_testsuite,
> and blockcipher suites.
> The unit test runner can then run the Cryptodev parent test suite,
> which in turn will run the sub-testsuites.
> 
> The user is now required to create vdevs via EAL commandline args,
> this is no longer done within the test app for crypto autotests.
> 
> Documentation will need to be added at a later stage,
> adding to the test document that isn't yet merged. [1]
> 
> [1] https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__patchwork.dpdk.org_project_dpdk_patch_20210309155757.615536-
> 2D1-2Daconole-
> 40redhat.com_&d=DwIDAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=DnL7Si2wl_PR
> wpZ9TWey3eu68gBzn7DkPwuqhd6WNyo&m=KnksoVYw74RhGrlmzJAVAS_O
> uxTVx-7GPFYdyP-qn14&s=uvfBln2-zGHif5yvsK0_QuZhdo4e-
> 3MpVXIevmc5OBE&e=
> 
> 
> v3:
>   - Added support for a testsuite having both a list of testcases,
>     and a list of sub-testsuites.
>   - Replaced PMD based parent testsuites with a cryptodev testsuite
>     used by all autotests, with the exception of scheduler autotest.
>   - Setup functions were added for all sub-testsuites, within which
>     required capability support is checked.
>   - The setup functions no longer create vdevs if needed,
>     this must be done by the user when running the test.
>   - Patch added to standardise return values for skipped testcases.
> v2:
>   - Added macro in place of testcase/testsuite loops.
>   - Added more detail in the summary output.
>   - Moved testcase counts to the testsuite structure.
>   - Flattened testsuite structure to remove union.
>   - Added patch for fix of blockcipher test return value.
>   - Squashed release note into last patch.
> 
> 
> Ciara Power (7):
>   app/test: refactor of unit test suite runner
>   test: introduce parent testsuite format
>   test/crypto: refactor to use sub-testsuites
>   test/crypto: replace unsupported with skipped
>   test/crypto: move testsuite params to header file
>   test/crypto: fix return value on test skipped
>   test/crypto: dynamically build blockcipher suite
> 
>  app/test/test.c                        |  176 +-
>  app/test/test.h                        |   23 +-
>  app/test/test_cryptodev.c              | 2324 ++++++++++++++----------
>  app/test/test_cryptodev.h              |   32 +
>  app/test/test_cryptodev_asym.c         |  111 +-
>  app/test/test_cryptodev_blockcipher.c  |  423 ++++-
>  app/test/test_cryptodev_blockcipher.h  |   12 +-
>  app/test/test_ipsec.c                  |   32 +-
>  doc/guides/rel_notes/release_21_05.rst |    5 +
>  9 files changed, 2000 insertions(+), 1138 deletions(-)
> 
> --
> 2.25.1


^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2021-05-08 15:38 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-23 16:18 [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework Ciara Power
2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 1/7] app/test: refactor of unit test suite runner Ciara Power
2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 2/7] test: introduce parent testsuite format Ciara Power
2021-04-23 16:32   ` Aaron Conole
2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 3/7] test/crypto: refactor to use sub-testsuites Ciara Power
2021-04-30 20:59   ` Doherty, Declan
2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 4/7] test/crypto: replace unsupported with skipped Ciara Power
2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 5/7] test/crypto: move testsuite params to header file Ciara Power
2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 6/7] test/crypto: fix return value on test skipped Ciara Power
2021-04-23 16:18 ` [dpdk-dev] [PATCH v3 7/7] test/crypto: dynamically build blockcipher suite Ciara Power
2021-04-25  9:24 ` [dpdk-dev] [PATCH v3 0/7] test: refactor crypto unit test framework Ruifeng Wang
2021-04-30 21:00 ` Doherty, Declan
2021-05-05 10:41   ` Hemant Agrawal
2021-05-05 12:20     ` Hemant Agrawal
2021-05-08 15:38 ` [dpdk-dev] [EXT] " Akhil Goyal

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).