DPDK CI discussions
 help / color / mirror / Atom feed
* [dpdk-ci] [v3] Expected results JSON format
@ 2018-03-20 13:56 Patrick MacArthur
  0 siblings, 0 replies; only message in thread
From: Patrick MacArthur @ 2018-03-20 13:56 UTC (permalink / raw)
  To: ci

Hi, all,

This is a slight revision based on my test implementation:

    {
      "results": [
        {
          "parameters": {
            "frame_size": 64,
            "txd/rxd": 1024
          },
          "throughput": {
            "result": "PASS",
            "delta": -0.452,
            "unit": "Mpps"
          }
        },
        /* ... */
      ]
    }

OR:

    {
      "results": [
        {
          "parameters": {
            "frame_size": 64,
            "txd/rxd": 1024
          },
         "throughput": {
            "result": "PASS",
            "actual": 45.783,
            "expected": 46.423,
            "unit": "Mpps"
          }
        },
        /* ... */
      ]
    }

The difference from the previous version that the input parameters are
now part of their own dictionary instead of loose within the "results"
dictionary.

Each entry in the results list is essentially a table row. The
parameters "frame_size" and "txd/rxd" are the input parameters for
each given measurement; what I gave here is just an example.

If the vendor script provides a delta, that delta is the only thing
that will be stored in the database for that test case. If the vendor
script provides actual and expected values, the expected value and the
computed delta will be stored in the database.

Either way, as discussed on the call, the results database API will
have access control to only allow access to data from the respective
vendor's users. Note that while we will endeavor to make our access
control as secure as possible, there is some inherent risk in any
database of a leak. Vendors should be aware of this potential risk and
weigh the advantage of having the absolute measurements accessible to
them against this potential risk.

Thanks,
Patrick

-- 
Patrick MacArthur
Research and Development, High Performance Networking and Storage
UNH InterOperability Laboratory

^ permalink raw reply	[flat|nested] only message in thread

only message in thread, other threads:[~2018-03-20 13:56 UTC | newest]

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-03-20 13:56 [dpdk-ci] [v3] Expected results JSON format Patrick MacArthur

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).