Patrick,

This is a great start. Thanks. Those of us supporting FPGA based NICs have the added challenge of almost a universe of different firmware/gateware - sometimes capable of being changed at runtime (e.g. partial reconfiguration). I think your schema as-is covers this; but I feel it would be better to additionally include a sha256 hash field for something like "Firmware Source ID". That is, every time we make new FPGA NIC Firmware (dozens of times per day), baked into the bitstream is the hash of the git commit that was used to produce that bitstream. This is enormously empowering, as it allows one later at run time, to determine exactly what the code was that created that image.

-Shep

Shepard Siegel, CTO
atomicrules.com


On Thu, Nov 9, 2017 at 6:53 PM, Patrick MacArthur <pmacarth@iol.unh.edu> wrote:
Hi, all,

I have been working on a database schema for storing performance
results. I am attempting to make this generic enough to store whatever
measurements we want.

I have attached an entity-relationship diagram of the schema which
should illustrate the starting point for the schema.

As a side note, I have managed to get DTS to run some of the
functional tests on a pair of nodes locally in a test setup while I
wait for equipment to be ready to ship. I am still working on a setup
to get it to run the performance tests so I can get some output to
parse to begin working on pulling information into the database.

Some notes on the tables:

patch: I propose that for this schema patches will be stored on the
filesystem content-addressable by the sha256 hash of the patch file.

patchset: "branch" refers to corresponding repo (master -> dpdk,
dpdk-next-net -> dpdk/next-net, etc.) and will be NULL until the
patchset is applied. Tarballs stored as files named after patchset id.

patchset_result: Result entry for a patchset. A patchset passes if
there is a result for every measurement which is either PASS or N/A.

environment: This should represent everything about where the test was
run and a new environment needs to be created every time this changes
(e.g., kernel or compiler update). I gathered the list of fields by
looking at the existing performance reports on the DPDK website. This
can be used for verification, to allow the test environment to be
reproducible, and to ensure that all comparisons are within an
identical setup.

measurement: A single measurement which can be applied to any
patchset. We can use values like (name: “BUILD”, higherIsBetter: TRUE,
expectedValue: 1, deltaLimit: 0) to verify non-performance conditions,
such as the build succeeding for the given environment.

The primary keys for these tables are not shown; I will likely be
implementing the database using the data modeling framework for
whatever Web backend we wind up selecting, which will set up primary
keys and table join relationships automatically.

Comments/suggestions? Is there anything that this schema does not cover?

Thanks,
Patrick

--
Patrick MacArthur
Research and Development, High Performance Networking and Storage
UNH InterOperability Laboratory