From: "Juraj Linkeš" <juraj.linkes@pantheon.tech>
To: thomas@monjalon.net, david.marchand@redhat.com,
Honnappa.Nagarahalli@arm.com, ohilyard@iol.unh.edu,
lijuan.tu@intel.com
Cc: dev@dpdk.org, "Juraj Linkeš" <juraj.linkes@pantheon.tech>
Subject: [RFC PATCH v1 20/24] dts: merge DTS doc/dts_gsg/usr_guide/results.rst to DPDK
Date: Wed, 6 Apr 2022 15:11:02 +0000 [thread overview]
Message-ID: <20220406151106.2915304-21-juraj.linkes@pantheon.tech> (raw)
In-Reply-To: <20220406151106.2915304-1-juraj.linkes@pantheon.tech>
---
dts/doc/dts_gsg/usr_guide/results.rst | 101 ++++++++++++++++++++++++++
1 file changed, 101 insertions(+)
create mode 100644 dts/doc/dts_gsg/usr_guide/results.rst
diff --git a/dts/doc/dts_gsg/usr_guide/results.rst b/dts/doc/dts_gsg/usr_guide/results.rst
new file mode 100644
index 0000000000..3ec08d86fe
--- /dev/null
+++ b/dts/doc/dts_gsg/usr_guide/results.rst
@@ -0,0 +1,101 @@
+Test Result
+===========
+
+Overview
+--------
+
+After DTS finished the validation, we can find the result files similar as below in output folder.
+
+.. code-block:: console
+
+ rst_report dts.log statistics.txt TestHelloWorld.log test_results.json test_results.xls
+
+* rst_report: contains the result RST file of performance data
+* dts.log: Full execution log of DTS framework
+* statstics.txt: summary statistics of DTS executed suites
+* TestHelloWorld.log: log message of Test suite: HelloWorld
+* test_result.json: json format result file
+* test_result.xls: excel format result file
+
+Statistics
+----------
+
+You can go through the summary of execution result via statistic.txt. This file includes the number of passed test cases, the number of failed case, the number of blocked and pass ratio.
+
+Please see example as the following. You can cat the sample file, then show this information of execution, totally executed two test cases, all cases passed the criterion and no failed or blocked cases.
+
+.. code-block:: console
+
+ [root@tester output]# cat statistics.txt
+ dpdk_version = 21.02.0
+ Passed = 2
+ Failed = 0
+ Blocked = 0
+ Pass rate = 100.0
+
+Details
+-------
+
+DTS provides 2 formats for test results, one is json, and the other is excel.
+If you need more detail information of test result, either of them is good to check.
+Both of them contain case names and results, also the failure reasons.
+
+* JSON result: result.json
+
+.. code-block:: console
+
+ {
+ "192.168.1.1": {
+ "dpdk_version": "21.02.0",
+ "nic": {
+ "driver": "vfio-pci",
+ "firmware": "8.00 0x80008c1a 1.2766.0",
+ "kdriver": "i40e-2.13.10",
+ "name": "fortville_25g"
+ },
+ "x86_64-native-linuxapp-gcc": {
+ "hello_world/test_hello_world_all_core": "passed"
+ "hello_world/test_hello_world_single_core": "passed"
+ }
+ }
+ }
+
+
+* Excel result: test_result.xls
+
+.. figure:: image/dts_result.png
+
+Logs
+----
+
+If you want to track more details about the process of each suite, please go to log file which named by this suite, all related information will stored in this file.
+
+Please see example for TestHelloWorld suite log as the following. This log file showed that application helloworld sent hello message from core1, and finally matched the pass criterion.
+
+.. code-block:: console
+
+ 31/12/2020 11:04:00 INFO:
+ TEST SUITE : TestHelloWorld
+ 31/12/2020 11:04:00 INFO: NIC : fortville_25g
+ 31/12/2020 11:04:00 SUITE_DUT_CMD: meson configure -Dexamples=helloworld x86_64-native-linuxapp-gcc
+ 31/12/2020 11:04:01 SUITE_DUT_CMD: ninja -C x86_64-native-linuxapp-gcc
+ 31/12/2020 11:04:07 SUITE_DUT_OUTPUT: ninja: Entering directory `x86_64-native-linuxapp-gcc'^M
+ [0/1] Regenerating build files.^M
+ The Meson build system^M
+ Version: 0.55.3^M
+ Source dir: /root/dpdk^M
+ Build dir: /root/dpdk/x86_64-native-linuxapp-gcc^M
+ Build type: native build^
+ …
+ Build targets in project: 998^M
+ Found ninja-1.10.0.git.kitware.jobserver-1 at /usr/local/bin/ninja^M
+ [1/2] Compiling C object examples/dpdk-helloworld.p/helloworld_main.c.o^M
+ [2/2] Linking target examples/dpdk-helloworld
+ 31/12/2020 11:04:09 INFO: Test Case test_hello_world_single_core Begin
+ 31/12/2020 11:04:13 SUITE_DUT_CMD: ./x86_64-native-linuxapp-gcc/examples/dpdk-helloworld -l 1 -n 4 --file-prefix=dpdk_10243_20201231110241
+ SUITE_DUT_OUTPUT: EAL: Detected 72 lcore(s)^M
+ …
+ hello from core 1
+ 31/12/2020 11:04:15 INFO: Test Case test_hello_world_single_core Result PASSED:
+ 31/12/2020 11:04:25 INFO:
+ TEST SUITE ENDED: TestHelloWorld
--
2.20.1
next prev parent reply other threads:[~2022-04-06 15:13 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-06 15:10 [RFC PATCH v1 00/24] merge DTS documentation files " Juraj Linkeš
2022-04-06 15:10 ` [RFC PATCH v1 01/24] dts: merge DTS CONTRIBUTING.TXT " Juraj Linkeš
2022-04-06 15:10 ` [RFC PATCH v1 02/24] dts: merge DTS doc/dts_gsg/Makefile " Juraj Linkeš
2022-04-06 15:10 ` [RFC PATCH v1 03/24] dts: merge DTS doc/dts_gsg/conf.py " Juraj Linkeš
2022-04-06 15:10 ` [RFC PATCH v1 04/24] dts: merge DTS doc/dts_gsg/image/dts_network_arch.png " Juraj Linkeš
2022-04-06 15:10 ` [RFC PATCH v1 05/24] dts: merge DTS doc/dts_gsg/image/dts_result.png " Juraj Linkeš
2022-04-06 15:10 ` [RFC PATCH v1 06/24] dts: merge DTS doc/dts_gsg/image/dts_soft_arch.png " Juraj Linkeš
2022-04-06 15:10 ` [RFC PATCH v1 07/24] dts: merge DTS doc/dts_gsg/image/scene_pf_passthrough.svg " Juraj Linkeš
2022-04-06 15:10 ` [RFC PATCH v1 08/24] dts: merge DTS doc/dts_gsg/image/virt_flow.svg " Juraj Linkeš
2022-04-06 15:10 ` [RFC PATCH v1 09/24] dts: merge DTS doc/dts_gsg/index.rst " Juraj Linkeš
2022-04-06 15:10 ` [RFC PATCH v1 10/24] dts: merge DTS doc/dts_gsg/quick_start.rst " Juraj Linkeš
2022-04-06 15:10 ` [RFC PATCH v1 11/24] dts: merge DTS doc/dts_gsg/rel_notes/index.rst " Juraj Linkeš
2022-04-06 15:10 ` [RFC PATCH v1 12/24] dts: merge DTS doc/dts_gsg/rel_notes/release_21_11.rst " Juraj Linkeš
2022-04-06 15:10 ` [RFC PATCH v1 13/24] dts: merge DTS doc/dts_gsg/usr_guide/asan_test.rst " Juraj Linkeš
2022-04-06 15:10 ` [RFC PATCH v1 14/24] dts: merge DTS doc/dts_gsg/usr_guide/igb_uio.rst " Juraj Linkeš
2022-04-06 15:10 ` [RFC PATCH v1 15/24] dts: merge DTS doc/dts_gsg/usr_guide/image/dts_func_deploy.png " Juraj Linkeš
2022-04-06 15:10 ` [RFC PATCH v1 16/24] dts: merge DTS doc/dts_gsg/usr_guide/image/dts_result.png " Juraj Linkeš
2022-04-06 15:10 ` [RFC PATCH v1 17/24] dts: merge DTS doc/dts_gsg/usr_guide/index.rst " Juraj Linkeš
2022-04-06 15:11 ` [RFC PATCH v1 18/24] dts: merge DTS doc/dts_gsg/usr_guide/intro.rst " Juraj Linkeš
2022-04-06 15:11 ` [RFC PATCH v1 19/24] dts: merge DTS doc/dts_gsg/usr_guide/ixia.rst " Juraj Linkeš
2022-04-06 15:11 ` Juraj Linkeš [this message]
2022-04-06 15:11 ` [RFC PATCH v1 21/24] dts: merge DTS doc/dts_gsg/usr_guide/setup_dut.rst " Juraj Linkeš
2022-04-06 15:11 ` [RFC PATCH v1 22/24] dts: merge DTS doc/dts_gsg/usr_guide/sys_reqs.rst " Juraj Linkeš
2022-04-06 15:11 ` [RFC PATCH v1 23/24] dts: merge DTS doc/dts_gsg/usr_guide/trex.rst " Juraj Linkeš
2022-04-06 15:11 ` [RFC PATCH v1 24/24] dts: merge DTS doc/dts_gsg/usr_guide/usage.rst " Juraj Linkeš
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220406151106.2915304-21-juraj.linkes@pantheon.tech \
--to=juraj.linkes@pantheon.tech \
--cc=Honnappa.Nagarahalli@arm.com \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=lijuan.tu@intel.com \
--cc=ohilyard@iol.unh.edu \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).