From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f42.google.com (mail-wm0-f42.google.com [74.125.82.42]) by dpdk.org (Postfix) with ESMTP id 97286379E for ; Wed, 30 Nov 2016 10:46:51 +0100 (CET) Received: by mail-wm0-f42.google.com with SMTP id u144so52700157wmu.1 for ; Wed, 30 Nov 2016 01:46:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:user-agent:in-reply-to :references:mime-version:content-transfer-encoding; bh=Wfec69IPdvYLnXPj0NM8yngXPC+PjSu5meDqLumPlIs=; b=cQqGwGbdKw6jn5GlWmfvP9RupBPHH6t8lUjQh7Ul1Dr+qiZZ2YdUGZQQKbyZ2HqcEV hwDkSSbrEJZr35cl4fgT+thCMByvdta64XL3YVZIR7catBuP/spa6vzoBH03844qRApk xlRTaTK03IyjCEaa8V5JFJN0cinPNUvzB+OZ1lCM/FjCoWKlCHuRqgubKV+iUUZTQDF8 LuWvEeyZiKg8pnzicUdodfHoiNG35m/N6XgGXWdCYMq3bZ8sef/lcIg0RE61Zc56hJWS W7dB50KZRI4m9v93fvlgnn/u/t8elabJvICJZ2CjAaACPkSoOCxTP0h3V87T5JgUU273 xz6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:user-agent :in-reply-to:references:mime-version:content-transfer-encoding; bh=Wfec69IPdvYLnXPj0NM8yngXPC+PjSu5meDqLumPlIs=; b=RIY5zyB/sTugUvoOw4lJr+SNGXbC9rjROiC3jRLt+w6J2AN4HI+1BKpUV2On7260GV +x+TzVdn1IUL2nso2Duo2e1Kmg3UgHt3dIH01rg2PwX0rLcBj+wr/ulXqqquxaAnNNnd GrzLOku0GtCCLr0vIb34mC400fhODaSZUPOUIg+W3OE7T6Q1PD3rR3qIkZF9J+jcIhiO Km1aWY9gh1cWA0GWrRYsnPsI5/OEta7AhTbRx4ZyLGYtWh7pZyHB5DYoJtr9OQXVeavr kikfUGnOrWVi9aEzrnuVxbsjVYOgqJtLTShQA7vn8jvfNXVJ8Wehf1XMo0Xw0B0ppe8c uMBg== X-Gm-Message-State: AKaTC02jazmf5KrVWeMFp255mcVz0DpPG0JuLsLixoHV/0dcQ1JT3CjmZLyig6m+1DXF2g86 X-Received: by 10.28.214.133 with SMTP id n127mr30213896wmg.28.1480499211130; Wed, 30 Nov 2016 01:46:51 -0800 (PST) Received: from xps13.localnet (184.203.134.77.rev.sfr.net. [77.134.203.184]) by smtp.gmail.com with ESMTPSA id pd2sm71984595wjb.31.2016.11.30.01.46.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 30 Nov 2016 01:46:50 -0800 (PST) From: Thomas Monjalon To: "Xu, Qian Q" Cc: ci@dpdk.org, "Wei, FangfangX" , "Liu, Yong" Date: Wed, 30 Nov 2016 10:46:49 +0100 Message-ID: <1683099.tRCFP3oIjm@xps13> User-Agent: KMail/4.14.10 (Linux/4.5.4-1-ARCH; KDE/4.14.11; x86_64; ; ) In-Reply-To: <82F45D86ADE5454A95A89742C8D1410E39278339@shsmsx102.ccr.corp.intel.com> References: <587443$9qu68@orsmga002.jf.intel.com> <2415876.bILSuWsmVC@xps13> <82F45D86ADE5454A95A89742C8D1410E39278339@shsmsx102.ccr.corp.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-ci] Intel PerPatch Build X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: CI discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Nov 2016 09:46:51 -0000 2016-11-30 09:25, Xu, Qian Q: > Then the conclusion is that we just kept current model, 1 report for 1 patch, right? Yes Thanks for the discussion (I keep this discussion in mind in case it needs some improvement) > Also keep in mind some things about the error comparison, we can discuss it again if we see many these issues. Yes > As to the next step, I think Fangfang can check how to update the per patch build result to the patchwork to make the result more visible/readable in website. Yes please. I'm also waiting to apply the patches. Should I apply them right now? > > -----Original Message----- > > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > > Sent: Wednesday, November 30, 2016 4:33 PM > > To: Xu, Qian Q > > Cc: ci@dpdk.org; Wei, FangfangX ; Liu, Yong > > > > Subject: Re: Intel PerPatch Build > > > > 2016-11-30 06:49, Xu, Qian Q: > > > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > > > > 2016-11-29 03:56, Xu, Qian Q: > > > > > I feel we need to split the report. > > > > > What do you think of having a report per OS? or a report per build? > > > > > It would show easily how big is the failure by looking at the > > > > > counters and > > > > descriptions in patchwork. > > > > > The email volume would be bigger but is it a problem? > > > > > ----current report is for per patch build report, and for each > > > > > patch, we now > > > > have 18 builds. We will send out 1 build report for 1 patch. If we > > > > send 18 reports for 1 patch, then it may be too many reports. > > > > > > > > Why is it too many reports? > > > > > > > For each patch, we have 18 builds tests and what you want is to have > > > 18 reports for each patch. If yes to it, normally we will have average ~30 > > patches(maybe not accurate) every day, then We will have 30x18=540 mails > > report every day, if we have 100 patches one day, then we will have 1800 > > reports. So I mean it would be too many reports. It would be a disaster for the > > mailing list. > > > I'm not sure if you like thousands of mails from one mailing list. Or do I > > misunderstand your points? > > > > I think this mailing-list does not aim to be human readable. > > The per-patch reports feed patchwork and the daily reports should feed another > > monitoring tool. > > That's why I wonder wether it is important to minimize the number of emails. > > > > > > In one build test, there can be several errors (even if we stop at the first > > error). > > > > And one error can be seen in several build tests. > > > > So we cannot really count the number of real errors, but we can > > > > count the number of tests which are failing. > > > > Yes we could add the number of failed tests in a test report. > > > > > > Yes, we can show the total numbers of failed tests in one test report, and it's > > also very simple. > > > > > > > I just thought that it would be simpler to understand if sending 1 report per > > test. > > > > > > Not sure if it's simpler but I can see mail spam if 1 report per test. > > > > It is not a spam if you are not registered to the mailing list :) > > > > > > An example of the benefit of splitting: > > > > An error with recent GCC is a failure. > > > > An error with ICC or an old GCC may be just a warning. > > > > > > Currently ICC and old GCC is not included in the per patch build system. > > > > > > > Finer is the grain of the reports, better is the patchwork overview. > > > > > > Agreed, but maybe there is other way. Fetch the failure numbers from the > > report. I also wonder if in future, there would be IBM or ARM build or even more > > regression functional test report. > > > If we put Intel IA build as 18 Lines for tracking how big the failure, > > > then IBM or ARM may have more lines, then how about functional test report? > > If we run 100 functional tests per patch, then We need 100 test reports for each > > patch, then if 100 patches for one day, then we have 10k mails one day...... > > > > > > From our monitor on the build error, not many errors currently, most > > > failures are due to the apply patch failures, and some > > > configurations(gcc-debug, gcc-combined) build errors. Build is relatively Stable > > now. So not sure how much value we can get from the split the patch report. > > Before we provide the patchset report, and now we have already split it to per > > patch report, which is also an Improved grain of the report. > > > > Yes good point. OK to continue with only one report for the Intel build tests as > > errors are rare. > > > > > > Today, we have few tests only. But later we could have more and more > > > > test instances sending the same kind of reports. > > > > > > > > Another example: > > > > If a test is failing for some reasons, it will fail for every patches. > > > > We must have a way to ignore it when reading the results. > > > > > > How to ignore it? Don't send the failure report?? > > > > I meant visually ignore it in the list of tests for a patch. > > But there is probably a better approach, like comparing with the results of the > > previous tests when making the report. > > I suggest to keep it in mind for later if we see this kind of issue. > > > > > We need analyze the failure reason then decide if the failure is same > > > for every patch, so the analysis is needed here. > > > > > > If it is a separate report, it is easier to ignore. > > > > > > > > > > > > > PS: please Qian, could you configure your email client to use reply quoting? > > > Yes, is it better now? > > > > Yes a lot better, thanks.