From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 22863A0559 for ; Mon, 16 Mar 2020 12:49:19 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 96CF51BF30; Mon, 16 Mar 2020 12:49:18 +0100 (CET) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) by dpdk.org (Postfix) with ESMTP id 883133B5; Mon, 16 Mar 2020 12:49:16 +0100 (CET) Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 187735C028A; Mon, 16 Mar 2020 07:49:16 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Mon, 16 Mar 2020 07:49:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=mesmtp; bh=ImXh3D6nUUHzs5tQMOv7t0uQNEdFUNxZBC5PtXVAz8k=; b=Na1X8UjIlcXO 15ojMG6A7xauvtvWAyicjqEkJJmiJzu7Sx0qzYwB5KfLcUPz/at4ZOuY1gHZuwZN 7p9OckpgoSdoMh30rn3uStAp/d5kcJuPsWxIGqvnmDOGblTRCwq9j8hjegPS3p/b rLZInMse/iLlrBUyWP0zjaTFKVDdICM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=ImXh3D6nUUHzs5tQMOv7t0uQNEdFUNxZBC5PtXVAz 8k=; b=LrUBVvHW2CKY42BzTnsZCDSAfpKYQ4CQqj8jZzPXJmFmsBLudYEw6X09q Vd97WoPGxwCUEm5GGQxrbvc7xO1UhbOtY+pc6n2z6fmcAAdhU5V6E1Fb4oLDiELI ozB7OqoShhMZO+69JanF2h360ICjbedLP+/t0E+qglbHk3tqlZEN+PXyvVgKVics bKXbYuIU4t3JjTC3G9vB400cSuwZ7d1aJjgOHqwMDnUf35fo776Kye0M5YOPokb7 iQvSia9pYDrwe6kaxnF0a0JOWFdMqwDbCkEszSExYp7F1mN3Joir/7z8RPbZesAK 2NZ3Q9n5oL0+s6cx+rYdKK8SKI7mQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedugedrudeffedgfeefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkfgjfhgggfgtsehtufertddttddvnecuhfhrohhmpefvhhhomhgr shcuofhonhhjrghlohhnuceothhhohhmrghssehmohhnjhgrlhhonhdrnhgvtheqnecuff homhgrihhnpeguphgukhdrohhrghdpuhhnhhdrvgguuhenucfkphepjeejrddufeegrddv tdefrddukeegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrh homhepthhhohhmrghssehmohhnjhgrlhhonhdrnhgvth X-ME-Proxy: Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA id E657C30618C1; Mon, 16 Mar 2020 07:49:14 -0400 (EDT) From: Thomas Monjalon To: "Richardson, Bruce" Cc: "web@dpdk.org" , "ci@dpdk.org" Date: Mon, 16 Mar 2020 12:49:13 +0100 Message-ID: <11237691.zAa99ISigo@xps> In-Reply-To: <59AF69C657FD0841A61C55336867B5B0975C93E4@IRSMSX103.ger.corp.intel.com> References: <20200310221130.46599-1-thomas@monjalon.net> <59AF69C657FD0841A61C55336867B5B0975C93E4@IRSMSX103.ger.corp.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-web] [PATCH] add general testing guidelines X-BeenThere: web@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK website maintenance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: web-bounces@dpdk.org Sender: "web" 16/03/2020 10:43, Richardson, Bruce: > From: web On Behalf Of Thomas Monjalon > > > > The community lab page is replaced with a more general page about testing, > > where the community lab is part of the big picture. It includes: > > - testing goals > > - CI workflow > > - tools > > General question - is this meant to describe the ideal state or the current state of testing? Yes it is about goals and workflow details, plus reference the tools which are used. I think these three items give a big picture. > It starts off largely inspirational, but then ends with lots of details on the current > setup. Do we want to have that mix in the one page? I plan to add one more page to list what is currently done in which lab. So the split is: - one guideline page to setup a lab - one status page I understand your question about splitting the guidelines in several pages but I don't whether it would help reading, or the opposite, makes harder to get all informations about what is a lab? > I've also put some stylistic comments inline below too. [...] > > +There are various ways of contributing to the CI effort: > > + > > +- suggest improvement > > +- write a new test > > +- improve infrastucture scripts > > Typo: infrastRucture OK thanks > > +- run a lab > > + > > +A specific effort is done > > +for the [community lab](//lab.dpdk.org/results/dashboard/about) > > +hosted at [UNH-IOL](//www.iol.unh.edu). > > + > > Not sure about the phrase "specific effort". How about "One specific lab instance has been set up for the community lab..." What about this? One specific lab instance, named community lab, is hosted at UNH-IOL. [...] > > +Ideally, the CI infrastructure should have these properties: > > + > > +- reliable > > +- neutral > > Neutral in what way? Presumably vendor or HW neutral? Neutral is a property which can apply to vendors, HW or SW (OS). Do you think "vendor neutral" is better? > > +- transparent visibility > > What is implied by this? It is "clear methodology" or "transparent testing methods"? I mean we should be able to see which tests are run, what is the environment, how tests are configured and what is the result. The property to measure is really the visibility. > > +- coverage summary > > +- trending graphs > > +- reports to author and > > +[test-report@dpdk.org](//inbox.dpdk.org/test-report/) > > +- easy to add a test > > +- easy to reproduce a test locally > > + > > The first three are what I'd consider properties, > but the rest are more functions that should be included in the CI, rather than properties of it. I don't agree: - coverage apply to all tests: what is the coverage of each test? - trending graphs can be done for any test or measure - reporting must be done for all tests - ability to add test is an infrastructure property - ability to reproduce is a test property [...] > > +The tests should cover these categories: > > + > > +- compilation > > +- static analysis > > +- functional > > +- performance > > +- interoperability > > +- portability > > +- compatibility > > + > > While compilation and static analysis are ok, the rest need "testing" after them since they are not nouns. Alternatively drop "static analysis" from the list and then change compilation to "compile" to use only adjectives. I don't want to drop static analysis, I think it is a test category. We could add "testing" after each item, but I feel it is just an overhead. I don't know what to do here. [...] > > +### Patch Testing > > + > > +When a patch is submitted, it is received via the mailing list > > +[dpdk-dev](//inbox.dpdk.org/dev/). > > +For simple tests, > > +like > > +[checkpatch](//git.dpdk.org/tools/dpdk-ci/tree/tests/checkpatch.sh), > > +it can be run directly on email receiving. > > s/email receiving/the received email/ I wanted to emphasize on the trigger which "receiving" instead of "polling". But "on the received email" is also OK. Is it a better English? Thanks for the review.