Re: [dpdk-ci] [RFC] test lab database schema

[Shepard Siegel <shepard.siegel@atomicrules.com>]

[-- Attachment #1: Type: text/plain, Size: 3272 bytes --]

Patrick,

This is a great start. Thanks. Those of us supporting FPGA based NICs have
the added challenge of almost a universe of different firmware/gateware -
sometimes capable of being changed at runtime (e.g. partial
reconfiguration). I think your schema as-is covers this; but I feel it
would be better to additionally include a sha256 hash field for something
like "Firmware Source ID". That is, every time we make new FPGA NIC
Firmware (dozens of times per day), baked into the bitstream is the hash of
the git commit that was used to produce that bitstream. This is enormously
empowering, as it allows one later at run time, to determine exactly what
the code was that created that image.

-Shep

Shepard Siegel, CTO
atomicrules.com


On Thu, Nov 9, 2017 at 6:53 PM, Patrick MacArthur <pmacarth@iol.unh.edu>
wrote:

> Hi, all,
>
> I have been working on a database schema for storing performance
> results. I am attempting to make this generic enough to store whatever
> measurements we want.
>
> I have attached an entity-relationship diagram of the schema which
> should illustrate the starting point for the schema.
>
> As a side note, I have managed to get DTS to run some of the
> functional tests on a pair of nodes locally in a test setup while I
> wait for equipment to be ready to ship. I am still working on a setup
> to get it to run the performance tests so I can get some output to
> parse to begin working on pulling information into the database.
>
> Some notes on the tables:
>
> patch: I propose that for this schema patches will be stored on the
> filesystem content-addressable by the sha256 hash of the patch file.
>
> patchset: "branch" refers to corresponding repo (master -> dpdk,
> dpdk-next-net -> dpdk/next-net, etc.) and will be NULL until the
> patchset is applied. Tarballs stored as files named after patchset id.
>
> patchset_result: Result entry for a patchset. A patchset passes if
> there is a result for every measurement which is either PASS or N/A.
>
> environment: This should represent everything about where the test was
> run and a new environment needs to be created every time this changes
> (e.g., kernel or compiler update). I gathered the list of fields by
> looking at the existing performance reports on the DPDK website. This
> can be used for verification, to allow the test environment to be
> reproducible, and to ensure that all comparisons are within an
> identical setup.
>
> measurement: A single measurement which can be applied to any
> patchset. We can use values like (name: “BUILD”, higherIsBetter: TRUE,
> expectedValue: 1, deltaLimit: 0) to verify non-performance conditions,
> such as the build succeeding for the given environment.
>
> The primary keys for these tables are not shown; I will likely be
> implementing the database using the data modeling framework for
> whatever Web backend we wind up selecting, which will set up primary
> keys and table join relationships automatically.
>
> Comments/suggestions? Is there anything that this schema does not cover?
>
> Thanks,
> Patrick
>
> --
> Patrick MacArthur
> Research and Development, High Performance Networking and Storage
> UNH InterOperability Laboratory
>

[-- Attachment #2: Type: text/html, Size: 3855 bytes --]