From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) by dpdk.org (Postfix) with ESMTP id 2C8D72B9E for ; Tue, 28 Nov 2017 15:14:19 +0100 (CET) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 2FB39209A1; Tue, 28 Nov 2017 09:14:18 -0500 (EST) Received: from frontend2 ([10.202.2.161]) by compute1.internal (MEProxy); Tue, 28 Nov 2017 09:14:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc; s=mesmtp; bh=iad+PJwtIBfI5rEm9RIKh1/tsZ nQAQp5cU5pSqXLX9E=; b=aYHSaEGuMDCitCAMQ3awEjV4gY5p4NH+jdRTGdt/xx etk0vmlunAtpCCZl5rRCuhgku5I6mZHiLZypyS4NQqvnrRKXTmmn/sVHX7xGol9o vBirKdmLIgjA38IoXwxjcIrJoG2GlTuOtt8gkm7wD8wR0yY80lAz+KHFJS/6Kt77 s= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=iad+PJ wtIBfI5rEm9RIKh1/tsZnQAQp5cU5pSqXLX9E=; b=DLNZ8JzUBJofCkw5/DF61m MpciK5xnFkdoRERsj4ERzO4UDXXLpy4oqsEQY4XhwV6IzxkK0YtXiVgWha4c1IV2 f0Aa4WKqhz4QhGZH/IUevuo24eUoT08H+iFF9dfoaShewLU3U9Z0efqbHLMMcQ0n 3RIC0bwM3qQowacROAxk3SERIduXlBVf0MXja66kkBfhY6GmkWovtFlQK78xFoiK 2UwQYB0fpIc47d4K97581Xeam/GEsVhMlF6e28TmnHl/Rw2mM6ovjm6gXkf9Uz+W UDN9m5G7dlPCw6Yuh50w0wMhJo2CocCcn1U8Sfw9EMgeB2fuD78rOtEmEyJcqkig == X-ME-Sender: Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA id D703324A6C; Tue, 28 Nov 2017 09:14:17 -0500 (EST) From: Thomas Monjalon To: Patrick MacArthur Cc: ci@dpdk.org, Bob Noseworthy Date: Tue, 28 Nov 2017 15:14:14 +0100 Message-ID: <2135477.CoD8K78SAQ@xps> In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-ci] [RFC] test lab database schema X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Nov 2017 14:14:19 -0000 Hi, 10/11/2017 00:53, Patrick MacArthur: > I have been working on a database schema for storing performance > results. I am attempting to make this generic enough to store whatever > measurements we want. Thanks for working on it. > I have attached an entity-relationship diagram of the schema which > should illustrate the starting point for the schema. I will do some comments below. [...] > patch: I propose that for this schema patches will be stored on the > filesystem content-addressable by the sha256 hash of the patch file. I don't see the need to store the full diff. You could store the patchwork id to retrieve it. > patchset: "branch" refers to corresponding repo (master -> dpdk, > dpdk-next-net -> dpdk/next-net, etc.) and will be NULL until the > patchset is applied. Tarballs stored as files named after patchset id. You should also be able to store measurements for a given state of a git tr= ee. It will be used for regular tests (like daily). > patchset_result: Result entry for a patchset. A patchset passes if > there is a result for every measurement which is either PASS or N/A. >=20 > environment: This should represent everything about where the test was > run and a new environment needs to be created every time this changes > (e.g., kernel or compiler update). I gathered the list of fields by > looking at the existing performance reports on the DPDK website. This > can be used for verification, to allow the test environment to be > reproducible, and to ensure that all comparisons are within an > identical setup. Do we want a limited set of environment fields, or something flexible? =46or instance, there is a field dts_config, but we could use other test to= ols. > measurement: A single measurement which can be applied to any > patchset. We can use values like (name: =E2=80=9CBUILD=E2=80=9D, higherIs= Better: TRUE, > expectedValue: 1, deltaLimit: 0) to verify non-performance conditions, > such as the build succeeding for the given environment. I think we should store the normal values per-environment at different time. I mean, the normal value will evolve with time and we should keep history o= f it. > The primary keys for these tables are not shown; I will likely be > implementing the database using the data modeling framework for > whatever Web backend we wind up selecting, which will set up primary > keys and table join relationships automatically. >=20 > Comments/suggestions? Is there anything that this schema does not cover? As I said above, we need to cover tests based on git state, not patchset. Thanks :)