[dpdk-ci] GA Baseline Performance Testing

[Jeremy Plsek <jplsek@iol.unh.edu>]

Hello all,

It was requested that we add a delta test result of the latest GA to
the result table. I said that it could be accomplished relatively
easily during the meeting. After some thought and laying out a plan,
adding this result will take much longer than expected.

If this ends up being a requirement before the results become public,
it may take much longer before the results become public.

Below are the steps that will have to be taken to get this working.

1. The member supplied test harness (such as DTS) will have to update
the `--update-expected` argument that expects an arbitrary value (such
as `master` or `v18.05.1`). This would save the absolute numbers as
before, but with appropriate keys. This will allow deltas to be
generated for each of those keys.
2. With the change above, the test harness will have to output a table
of deltas for each key defined.
3. Add an option to delete said keys (such as `--delete-expected
v17.11.4`) so we don’t have to generate more tables than needed.
4. Update the database to expect the individual delta results to
contain these tags. Old results would be migrated to a `master` tag.
5. Update the dashboard to provide a table of results for each unique tag.
6. Update current jobs with the new test harness parameters (which
would use `--update-expected master`, for example).
7. Create a new script that finds the latest tag from dpdk-stable,
builds, and uploads to our database appropriately (which would use
`--update-expected v18.05.1`, for example). Then delete older
"expected" keys.
8. Create GA performance Jenkins Job to run this script.
9. Done? (Everything should be automatic at this point?)

Implementation specifics will be added to the policy document if
members would like to go through with this.

This may also help pave the way for comparing GA versions as well (as
was also requested). Currently, with the data we have, we can compare
the performance delta of master. (I made a quick-and-dirty script that
plotted master over time before the presentation at the Summit, but my
logic was slightly off, so I didn’t present it.)

If anyone has a better way of doing this, please feel free to provide
suggestions.

I estimate that this will take at least a month to add these delta results.

Thanks
-- 
Jeremy Plsek
UNH InterOperability Laboratory