From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f41.google.com (mail-wm0-f41.google.com [74.125.82.41]) by dpdk.org (Postfix) with ESMTP id D4C433B5 for ; Tue, 29 Nov 2016 10:21:00 +0100 (CET) Received: by mail-wm0-f41.google.com with SMTP id t79so180921562wmt.0 for ; Tue, 29 Nov 2016 01:21:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:user-agent:in-reply-to :references:mime-version:content-transfer-encoding; bh=GSAw9lTnhaayfFp00KvK+NVfHAjohEwgSohywQTyfic=; b=fXWGbU0iGpUOcl/W+A4fUxFH0NVyH1ETAJFa10EASiPjtb8cnENI/SpBan8x/Doloy Xa7CG5L2kePArUbZ8wUU6yXHWDT3Ua7wWWBoyIgnc1icTbNAQcYl3ucKH3AtNWnJNQpI F/9cQ6WIoaMxXNhlUyuMfsd9mR8SMjYD5dHP7BKPfra+5D5aUhEajft+Bhob+UXrjexc bB3w2OU9AgKpDRy1f5v0aQvtg+WEA9nui409bjeJwpjSUC3pSkmSwUIQ7hePO9JfJs+P At/6Js6KOhMGs59WAFE3y4DctOPEVvKvKNBC3uezPjjIAMzCIyfhPv2ALB+AM7EWMdWo BhEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:user-agent :in-reply-to:references:mime-version:content-transfer-encoding; bh=GSAw9lTnhaayfFp00KvK+NVfHAjohEwgSohywQTyfic=; b=BFIvnY0yGnSARRpzG7WhjevZ9MzHxwIH0eH2JSppS+uLjjlxuY8LP9CYqZ8yPEm5ay kQCacFlN/rbgR00g40eCtFkNs8XoYN0Q64nV9DmamfdvijIOGbLcKt1ugTFvKf77YL10 7jyTe+aWkhgKNziNNO5tEX/LdFnMsoxKn1jDDSZOqBh2APETrXSXDADq03FMiTpmAXmu i6SNV021NhrIUsapM+LgoI8mGVCsK7YZQ+VnWsUkEqRz+6n9QdN8cWlq2Mejss1f4Qxy BkjHfhk7MA4s83siz4R/0u34lY1dtHT+1CDbVkXl31Mm1qLy8JxOI+Tu1KfBhbfoVGlt M+3g== X-Gm-Message-State: AKaTC02fLF55p9AhgepVpZbUC5PGYHWlO0Ngss8chvxLQRIV8GyjhDU4Nwvpx/ga6ZSu4Q7G X-Received: by 10.28.215.6 with SMTP id o6mr22472357wmg.5.1480411260558; Tue, 29 Nov 2016 01:21:00 -0800 (PST) Received: from xps13.localnet (184.203.134.77.rev.sfr.net. [77.134.203.184]) by smtp.gmail.com with ESMTPSA id js10sm66705129wjb.19.2016.11.29.01.20.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Nov 2016 01:20:59 -0800 (PST) From: Thomas Monjalon To: "Xu, Qian Q" Cc: ci@dpdk.org, "Wei, FangfangX" , "Liu, Yong" Date: Tue, 29 Nov 2016 10:20:59 +0100 Message-ID: <8972903.ZmvP4EtD2m@xps13> User-Agent: KMail/4.14.10 (Linux/4.5.4-1-ARCH; KDE/4.14.11; x86_64; ; ) In-Reply-To: <82F45D86ADE5454A95A89742C8D1410E3927590F@shsmsx102.ccr.corp.intel.com> References: <587443$9qu68@orsmga002.jf.intel.com> <3110646.SlUphBRNZp@xps13> <82F45D86ADE5454A95A89742C8D1410E3927590F@shsmsx102.ccr.corp.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-ci] Intel PerPatch Build X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Nov 2016 09:21:01 -0000 2016-11-29 03:56, Xu, Qian Q: > I feel we need to split the report. > What do you think of having a report per OS? or a report per build? > It would show easily how big is the failure by looking at the counters and descriptions in patchwork. > The email volume would be bigger but is it a problem? > ----current report is for per patch build report, and for each patch, we now have 18 builds. We will send out 1 build report for 1 patch. If we send 18 reports for 1 patch, then it may be too many reports. Why is it too many reports? > If you want to check how many failures for the build, for example, 18 builds, then 1 failures, there is 2 ways to check. > 1. you can get how many failures information from the Build Summary. > > Patch17274-17274 --> compile pass > > Build Summary: 18 Builds Done, 18 Successful, 0 Failures > > 2. We can add the failure numbers in the title, such as [dpdk-test-report] |2 FAILURE|xxxxx > > Does it make sense? In one build test, there can be several errors (even if we stop at the first error). And one error can be seen in several build tests. So we cannot really count the number of real errors, but we can count the number of tests which are failing. Yes we could add the number of failed tests in a test report. I just thought that it would be simpler to understand if sending 1 report per test. An example of the benefit of splitting: An error with recent GCC is a failure. An error with ICC or an old GCC may be just a warning. Finer is the grain of the reports, better is the patchwork overview. Today, we have few tests only. But later we could have more and more test instances sending the same kind of reports. Another example: If a test is failing for some reasons, it will fail for every patches. We must have a way to ignore it when reading the results. If it is a separate report, it is easier to ignore. > If I understand well you test every patches of a series at once and send the report for the last patch of the series? > ----No. We send the report for each patch of the series, not the last one. The process of per patch build check is as below. > 1. We pull the patchset, for example, we have 10 patches in 1 patchset. We will apply all the patches to the git tree. > If apply failed for one tree, we will try other trees(such as next-virtio, next-crypto, next-net), if any tree can be applied, then > We think the patch apply OK. If all trees can't be applied, we take it as Failure. Then we will send out 10 patch reports, each one is Failure. > > 2. If apply patch is OK, then we do per patch build check one by one. After 10 patch's build done, we will send report one by one. > For example, 10 patches in 1 patchset, we will do 18 builds for each patch, totally 180 builds for the patchset. Then send report for each patch after the last patch. > Test1: patch1: 18 build > Test2: patch1 + patch2: 18 build > .... > Test10: patch1+xxx+patch10: 18build > > Is it clear? OK perfect, thanks PS: please Qian, could you configure your email client to use reply quoting?