Re: [dpdk-ci] DPDK Lab

Re: [dpdk-ci] DPDK Lab

[Xu, Qian Q]

To do the per patchset performance test, we need to know that the patchset's target repo. Any thoughts here? 


> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> Sent: Thursday, April 6, 2017 7:46 PM
> To: Xu, Qian Q <qian.q.xu@intel.com>
> Cc: Erez Scop <erezsc@mellanox.com>; O'Driscoll, Tim
> <tim.odriscoll@intel.com>; Liu, Yong <yong.liu@intel.com>; 'Vincent JARDIN'
> <vincent.jardin@6wind.com>; Guillaume Gaudonville
> <Guillaume.Gaudonville@6wind.com>; Shreyansh Jain
> <shreyansh.jain@nxp.com>; 'Kevin Traynor' <ktraynor@redhat.com>
> Subject: Re: DPDK Lab
> 
> 2017-04-06 09:57, Xu, Qian Q:
> > As to the per patchset performance test, as you know, there are many repos
> for DPDK now, so when a new patchset coming, which repo should it go to?
> Maybe it doesn't go to dpdk master, but go to dpdk-next-virtio, or others, but
> automation tool doesn't know that.
> > As to per patch build, we are trying the patch on different repo to find the one
> that can pass build, but as to performance test for per patchset, we must apply
> the patchset into correct repo/branch or else the code didn't work.
> > Any comments here?
> 
> For most of the patches it should be obvious to determine the targetted repo.
> We should add a script in dpdk-ci to get the repo name from a patchset.
> Anyway, this technical discussion should be on ci@dpdk.org.

Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

[Thomas Monjalon]

2017-04-13 05:25, Xu, Qian Q:
> To do the per patchset performance test, we need to know that the patchset's target repo. Any thoughts here? 

The first task is to extract modified files from lines starting with +++.
Then we must apply some basic rules.
Example: if every paths start with 
	drivers/crypto/
	or lib/librte_cryptodev/
	or doc/guides/cryptodevs/
	then it goes to dpdk-next-crypto

Later it would be a good idea to store the rules in the dpdk repo near
the file MAINTAINERS, so patchwork could parse them and display the
maintainers and the tree for the patches.


> > -----Original Message-----
> > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> > Sent: Thursday, April 6, 2017 7:46 PM
> > 
> > 2017-04-06 09:57, Xu, Qian Q:
> > > As to the per patchset performance test, as you know, there are many repos
> > for DPDK now, so when a new patchset coming, which repo should it go to?
> > Maybe it doesn't go to dpdk master, but go to dpdk-next-virtio, or others, but
> > automation tool doesn't know that.
> > > As to per patch build, we are trying the patch on different repo to find the one
> > that can pass build, but as to performance test for per patchset, we must apply
> > the patchset into correct repo/branch or else the code didn't work.
> > > Any comments here?
> > 
> > For most of the patches it should be obvious to determine the targetted repo.
> > We should add a script in dpdk-ci to get the repo name from a patchset.
> > Anyway, this technical discussion should be on ci@dpdk.org.

Re: [dpdk-ci] DPDK Lab

[Richardson, Bruce]

Majority (90%) of the time, the subject prefix on the patches should provide the hint. If the majority of the patches start with "net/" then it's for next-net, and similarly for crypto. Patches with virtio or vhost go to that tree, etc. There will be edge cases where we get it wrong or just can't automatically determine the tree, but those patchset should perhaps have delayed checking after we have manual assignment of tree. What could also work in those cases is to wait to scan them until we get a delegate assigned in patchwork, as that will tell us what tree they are to hit.

/Bruce

> -----Original Message-----
> From: Xu, Qian Q
> Sent: Thursday, April 13, 2017 6:26 AM
> To: ci@dpdk.org
> Cc: Richardson, Bruce <bruce.richardson@intel.com>; Xu, Qian Q
> <qian.q.xu@intel.com>; Thomas Monjalon <thomas.monjalon@6wind.com>
> Subject: RE: DPDK Lab
> 
> To do the per patchset performance test, we need to know that the
> patchset's target repo. Any thoughts here?
> 
> 
> > -----Original Message-----
> > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> > Sent: Thursday, April 6, 2017 7:46 PM
> > To: Xu, Qian Q <qian.q.xu@intel.com>
> > Cc: Erez Scop <erezsc@mellanox.com>; O'Driscoll, Tim
> > <tim.odriscoll@intel.com>; Liu, Yong <yong.liu@intel.com>; 'Vincent
> JARDIN'
> > <vincent.jardin@6wind.com>; Guillaume Gaudonville
> > <Guillaume.Gaudonville@6wind.com>; Shreyansh Jain
> > <shreyansh.jain@nxp.com>; 'Kevin Traynor' <ktraynor@redhat.com>
> > Subject: Re: DPDK Lab
> >
> > 2017-04-06 09:57, Xu, Qian Q:
> > > As to the per patchset performance test, as you know, there are many
> > > repos
> > for DPDK now, so when a new patchset coming, which repo should it go to?
> > Maybe it doesn't go to dpdk master, but go to dpdk-next-virtio, or
> > others, but automation tool doesn't know that.
> > > As to per patch build, we are trying the patch on different repo to
> > > find the one
> > that can pass build, but as to performance test for per patchset, we
> > must apply the patchset into correct repo/branch or else the code didn't
> work.
> > > Any comments here?
> >
> > For most of the patches it should be obvious to determine the targetted
> repo.
> > We should add a script in dpdk-ci to get the repo name from a patchset.
> > Anyway, this technical discussion should be on ci@dpdk.org.

Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

[Xu, Qian Q]

Thanks Thomas and Bruce for your comments. Currently we are trying to start the per patch set performance test investigation and need to know the 
Target repo for each patch set.  
The goal is to get 100% correct repo to go for each patch set. It's important for the credit of the performance result. 
@Thomas, could you provide any script to support the auto-filter after the 17.05 release? Currently, we don't have the ability to make it. 

> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> Sent: Thursday, April 13, 2017 2:46 PM
> To: Xu, Qian Q <qian.q.xu@intel.com>
> Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>
> Subject: Re: script to determine target repo (was DPDK Lab)
> 
> 2017-04-13 05:25, Xu, Qian Q:
> > To do the per patchset performance test, we need to know that the patchset's
> target repo. Any thoughts here?
> 
> The first task is to extract modified files from lines starting with +++.
> Then we must apply some basic rules.
> Example: if every paths start with
> 	drivers/crypto/
> 	or lib/librte_cryptodev/
> 	or doc/guides/cryptodevs/
> 	then it goes to dpdk-next-crypto
> 
> Later it would be a good idea to store the rules in the dpdk repo near the file
> MAINTAINERS, so patchwork could parse them and display the maintainers and
> the tree for the patches.
> 
> 
> > > -----Original Message-----
> > > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> > > Sent: Thursday, April 6, 2017 7:46 PM
> > >
> > > 2017-04-06 09:57, Xu, Qian Q:
> > > > As to the per patchset performance test, as you know, there are
> > > > many repos
> > > for DPDK now, so when a new patchset coming, which repo should it go to?
> > > Maybe it doesn't go to dpdk master, but go to dpdk-next-virtio, or
> > > others, but automation tool doesn't know that.
> > > > As to per patch build, we are trying the patch on different repo
> > > > to find the one
> > > that can pass build, but as to performance test for per patchset, we
> > > must apply the patchset into correct repo/branch or else the code didn't
> work.
> > > > Any comments here?
> > >
> > > For most of the patches it should be obvious to determine the targetted repo.
> > > We should add a script in dpdk-ci to get the repo name from a patchset.
> > > Anyway, this technical discussion should be on ci@dpdk.org.
> 

Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

[Xu, Qian Q]

Sent to correct Thomas' mail:), Thomas, could you see below? Thx. 

> -----Original Message-----
> From: Xu, Qian Q
> Sent: Tuesday, April 25, 2017 2:24 PM
> To: Thomas Monjalon <thomas.monjalon@6wind.com>
> Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; Wei,
> FangfangX <fangfangx.wei@intel.com>; O'Driscoll, Tim
> <tim.odriscoll@intel.com>; Liu, Yuanhan <yuanhan.liu@intel.com>; Xu, Qian Q
> <qian.q.xu@intel.com>
> Subject: RE: script to determine target repo (was DPDK Lab)
> 
> Thanks Thomas and Bruce for your comments. Currently we are trying to start
> the per patch set performance test investigation and need to know the Target
> repo for each patch set.
> The goal is to get 100% correct repo to go for each patch set. It's important for
> the credit of the performance result.
> @Thomas, could you provide any script to support the auto-filter after the 17.05
> release? Currently, we don't have the ability to make it.
> 
> > -----Original Message-----
> > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> > Sent: Thursday, April 13, 2017 2:46 PM
> > To: Xu, Qian Q <qian.q.xu@intel.com>
> > Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>
> > Subject: Re: script to determine target repo (was DPDK Lab)
> >
> > 2017-04-13 05:25, Xu, Qian Q:
> > > To do the per patchset performance test, we need to know that the
> > > patchset's
> > target repo. Any thoughts here?
> >
> > The first task is to extract modified files from lines starting with +++.
> > Then we must apply some basic rules.
> > Example: if every paths start with
> > 	drivers/crypto/
> > 	or lib/librte_cryptodev/
> > 	or doc/guides/cryptodevs/
> > 	then it goes to dpdk-next-crypto
> >
> > Later it would be a good idea to store the rules in the dpdk repo near
> > the file MAINTAINERS, so patchwork could parse them and display the
> > maintainers and the tree for the patches.
> >
> >
> > > > -----Original Message-----
> > > > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> > > > Sent: Thursday, April 6, 2017 7:46 PM
> > > >
> > > > 2017-04-06 09:57, Xu, Qian Q:
> > > > > As to the per patchset performance test, as you know, there are
> > > > > many repos
> > > > for DPDK now, so when a new patchset coming, which repo should it go to?
> > > > Maybe it doesn't go to dpdk master, but go to dpdk-next-virtio, or
> > > > others, but automation tool doesn't know that.
> > > > > As to per patch build, we are trying the patch on different repo
> > > > > to find the one
> > > > that can pass build, but as to performance test for per patchset,
> > > > we must apply the patchset into correct repo/branch or else the
> > > > code didn't
> > work.
> > > > > Any comments here?
> > > >
> > > > For most of the patches it should be obvious to determine the targetted
> repo.
> > > > We should add a script in dpdk-ci to get the repo name from a patchset.
> > > > Anyway, this technical discussion should be on ci@dpdk.org.
> >

Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

[Wei, FangfangX]

Hi Thomas,
Another question need your help: How I can auto-filter the patchset which include some patches?
Currently we filter the patchset with the format 'Message-Id: <([0-9-]+)-git-send-email-.*@.*>' in per-patch build testing system. But I find that there are some different format about the message-id, such as " 20170420072402.38106-5-aik@ozlabs.ru", "1492658865-30077-1-git-send-email-wei.zhao1@intel.com", "20170209165959.28082-3-benjamin.walker@intel.com", " 3da244231b6868ccc3e70d91dc277df39a3695bd.1481592081.git.mirq-linux@rere.qmqm.pl" and etc.


Best Regards
Fangfang Wei

-----Original Message-----
From: Xu, Qian Q 
Sent: Tuesday, April 25, 2017 2:24 PM
To: Thomas Monjalon <thomas.monjalon@6wind.com>
Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; Wei, FangfangX <fangfangx.wei@intel.com>; O'Driscoll, Tim <tim.odriscoll@intel.com>; Liu, Yuanhan <yuanhan.liu@intel.com>; Xu, Qian Q <qian.q.xu@intel.com>
Subject: RE: script to determine target repo (was DPDK Lab)

Thanks Thomas and Bruce for your comments. Currently we are trying to start the per patch set performance test investigation and need to know the Target repo for each patch set.  
The goal is to get 100% correct repo to go for each patch set. It's important for the credit of the performance result. 
@Thomas, could you provide any script to support the auto-filter after the 17.05 release? Currently, we don't have the ability to make it. 

> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> Sent: Thursday, April 13, 2017 2:46 PM
> To: Xu, Qian Q <qian.q.xu@intel.com>
> Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>
> Subject: Re: script to determine target repo (was DPDK Lab)
> 
> 2017-04-13 05:25, Xu, Qian Q:
> > To do the per patchset performance test, we need to know that the 
> > patchset's
> target repo. Any thoughts here?
> 
> The first task is to extract modified files from lines starting with +++.
> Then we must apply some basic rules.
> Example: if every paths start with
> 	drivers/crypto/
> 	or lib/librte_cryptodev/
> 	or doc/guides/cryptodevs/
> 	then it goes to dpdk-next-crypto
> 
> Later it would be a good idea to store the rules in the dpdk repo near 
> the file MAINTAINERS, so patchwork could parse them and display the 
> maintainers and the tree for the patches.
> 
> 
> > > -----Original Message-----
> > > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> > > Sent: Thursday, April 6, 2017 7:46 PM
> > >
> > > 2017-04-06 09:57, Xu, Qian Q:
> > > > As to the per patchset performance test, as you know, there are 
> > > > many repos
> > > for DPDK now, so when a new patchset coming, which repo should it go to?
> > > Maybe it doesn't go to dpdk master, but go to dpdk-next-virtio, or 
> > > others, but automation tool doesn't know that.
> > > > As to per patch build, we are trying the patch on different repo 
> > > > to find the one
> > > that can pass build, but as to performance test for per patchset, 
> > > we must apply the patchset into correct repo/branch or else the 
> > > code didn't
> work.
> > > > Any comments here?
> > >
> > > For most of the patches it should be obvious to determine the targetted repo.
> > > We should add a script in dpdk-ci to get the repo name from a patchset.
> > > Anyway, this technical discussion should be on ci@dpdk.org.
> 

Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

[Wei, FangfangX]

Sorry for wrong email address.

-----Original Message-----
From: ci [mailto:ci-bounces@dpdk.org] On Behalf Of Wei, FangfangX
Sent: Tuesday, April 25, 2017 2:38 PM
To: Xu, Qian Q <qian.q.xu@intel.com>; Thomas Monjalon <thomas.monjalon@6wind.com>
Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; O'Driscoll, Tim <tim.odriscoll@intel.com>; Liu, Yuanhan <yuanhan.liu@intel.com>
Subject: Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

Hi Thomas,
Another question need your help: How I can auto-filter the patchset which include some patches?
Currently we filter the patchset with the format 'Message-Id: <([0-9-]+)-git-send-email-.*@.*>' in per-patch build testing system. But I find that there are some different format about the message-id, such as " 20170420072402.38106-5-aik@ozlabs.ru", "1492658865-30077-1-git-send-email-wei.zhao1@intel.com", "20170209165959.28082-3-benjamin.walker@intel.com", " 3da244231b6868ccc3e70d91dc277df39a3695bd.1481592081.git.mirq-linux@rere.qmqm.pl" and etc.


Best Regards
Fangfang Wei

-----Original Message-----
From: Xu, Qian Q
Sent: Tuesday, April 25, 2017 2:24 PM
To: Thomas Monjalon <thomas.monjalon@6wind.com>
Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; Wei, FangfangX <fangfangx.wei@intel.com>; O'Driscoll, Tim <tim.odriscoll@intel.com>; Liu, Yuanhan <yuanhan.liu@intel.com>; Xu, Qian Q <qian.q.xu@intel.com>
Subject: RE: script to determine target repo (was DPDK Lab)

Thanks Thomas and Bruce for your comments. Currently we are trying to start the per patch set performance test investigation and need to know the Target repo for each patch set.  
The goal is to get 100% correct repo to go for each patch set. It's important for the credit of the performance result. 
@Thomas, could you provide any script to support the auto-filter after the 17.05 release? Currently, we don't have the ability to make it. 

> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> Sent: Thursday, April 13, 2017 2:46 PM
> To: Xu, Qian Q <qian.q.xu@intel.com>
> Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>
> Subject: Re: script to determine target repo (was DPDK Lab)
> 
> 2017-04-13 05:25, Xu, Qian Q:
> > To do the per patchset performance test, we need to know that the 
> > patchset's
> target repo. Any thoughts here?
> 
> The first task is to extract modified files from lines starting with +++.
> Then we must apply some basic rules.
> Example: if every paths start with
> 	drivers/crypto/
> 	or lib/librte_cryptodev/
> 	or doc/guides/cryptodevs/
> 	then it goes to dpdk-next-crypto
> 
> Later it would be a good idea to store the rules in the dpdk repo near 
> the file MAINTAINERS, so patchwork could parse them and display the 
> maintainers and the tree for the patches.
> 
> 
> > > -----Original Message-----
> > > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> > > Sent: Thursday, April 6, 2017 7:46 PM
> > >
> > > 2017-04-06 09:57, Xu, Qian Q:
> > > > As to the per patchset performance test, as you know, there are 
> > > > many repos
> > > for DPDK now, so when a new patchset coming, which repo should it go to?
> > > Maybe it doesn't go to dpdk master, but go to dpdk-next-virtio, or 
> > > others, but automation tool doesn't know that.
> > > > As to per patch build, we are trying the patch on different repo 
> > > > to find the one
> > > that can pass build, but as to performance test for per patchset, 
> > > we must apply the patchset into correct repo/branch or else the 
> > > code didn't
> work.
> > > > Any comments here?
> > >
> > > For most of the patches it should be obvious to determine the targetted repo.
> > > We should add a script in dpdk-ci to get the repo name from a patchset.
> > > Anyway, this technical discussion should be on ci@dpdk.org.
> 

Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

[Thomas Monjalon]

Hi,

25/04/2017 08:37, Wei, FangfangX:
> Hi Thomas,
> Another question need your help: How I can auto-filter the patchset which include some patches?
> Currently we filter the patchset with the format 'Message-Id: <([0-9-]+)-git-send-email-.*@.*>' in per-patch build testing system. But I find that there are some different format about the message-id, such as " 20170420072402.38106-5-aik@ozlabs.ru", "1492658865-30077-1-git-send-email-wei.zhao1@intel.com", "20170209165959.28082-3-benjamin.walker@intel.com", " 3da244231b6868ccc3e70d91dc277df39a3695bd.1481592081.git.mirq-linux@rere.qmqm.pl" and etc.

In the dpdk-ci repo, there is a script tools/filter-patch-email.sh.

Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

[Thomas Monjalon]

25/04/2017 08:23, Xu, Qian Q:
> Thanks Thomas and Bruce for your comments. Currently we are trying to start the per patch set performance test investigation and need to know the 
> Target repo for each patch set.  
> The goal is to get 100% correct repo to go for each patch set. It's important for the credit of the performance result. 
> @Thomas, could you provide any script to support the auto-filter after the 17.05 release? Currently, we don't have the ability to make it. 

OK, it is on my todo list.

Other task: I want to check the series support in the patchwork git.

Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

[Wei, FangfangX]

Hi Thomas,

What's the status about the script which determine target repo for each patch set?
Currently I used the rule which you told me before: "the subject prefix on the patches should provide the hint. If the majority of the patches start with "net/" then it's for next-net, and similarly for crypto. Patches with virtio or vhost go to that tree, etc".
But I'm not sure about the 100% correct repo.
Looking forward to your script.

Another thing, Qian told me that you want to know the apply sequence about many patches in one patch set.
I don't apply patches with the order of patchwork id, but the names' order about them.
Such as, patchset 25081-25085, the order of patchwork id is not the correct patches' order.
http://dpdk.org/patch/25081, subject: [dpdk-dev,v2,1/5] net/qede: refactoring vport handling code
http://dpdk.org/patch/25083, subject: [dpdk-dev,v2,2/5] net/qede: refactoring multi-queue implementation
http://dpdk.org/patch/25084, subject: [dpdk-dev,v2,3/5] net/qede/base: upgrade the FW to 8.20.0.0
http://dpdk.org/patch/25082, subject: [dpdk-dev,v2,4/5] net/qede: fix VXLAN tunnel Tx offload flag setting
http://dpdk.org/patch/25085, subject: [dpdk-dev,v2,5/5] net/qede: refactor Tx routine
After download them, the patches' names are as below:
http://dpdk.org/patch/25081: dpdk-dev-v2-1-5-net-qede-refactoring-vport-handling-code.patch
http://dpdk.org/patch/25083: dpdk-dev-v2-2-5-net-qede-refactoring-multi-queue-implementation.patch
http://dpdk.org/patch/25084: dpdk-dev-v2-3-5-net-qede-base-upgrade-the-FW-to-8.20.0.0.patch
http://dpdk.org/patch/25082: dpdk-dev-v2-4-5-net-qede-fix-VXLAN-tunnel-Tx-offload-flag-setting.patch
http://dpdk.org/patch/25085: dpdk-dev-v2-5-5-net-qede-refactor-Tx-routine.patch

So applying these patches based on the order of their names, it will be correct.


Best Regards
Fangfang Wei


-----Original Message-----
From: Thomas Monjalon [mailto:thomas@monjalon.net] 
Sent: Tuesday, April 25, 2017 2:51 PM
To: Xu, Qian Q <qian.q.xu@intel.com>
Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; Wei, FangfangX <fangfangx.wei@intel.com>; O'Driscoll, Tim <tim.odriscoll@intel.com>; Liu, Yuanhan <yuanhan.liu@intel.com>
Subject: Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

25/04/2017 08:23, Xu, Qian Q:
> Thanks Thomas and Bruce for your comments. Currently we are trying to 
> start the per patch set performance test investigation and need to know the Target repo for each patch set.
> The goal is to get 100% correct repo to go for each patch set. It's important for the credit of the performance result. 
> @Thomas, could you provide any script to support the auto-filter after the 17.05 release? Currently, we don't have the ability to make it. 

OK, it is on my todo list.

Other task: I want to check the series support in the patchwork git.

Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

[Richardson, Bruce]

Hi Fangfang,

My comments on the questions you asked:

1. For determining the repo tree to target, I don't believe that we can ever come up with a 100% accurate rule, as the tree to which a set is to be applied can be difficult to determine, so it may be done on the basis of on-list discussion. A 90% accurate rule it what we may have to accept. However, since applying a patchset to a tree should not be a time-consuming operation, I suggest any script produce a list of possible trees in priority order to try. If not net, then try main, etc. etc.

2. Using the order from patch titles is correct.

/Bruce

> -----Original Message-----
> From: Wei, FangfangX
> Sent: Wednesday, June 21, 2017 8:46 AM
> To: Thomas Monjalon <thomas@monjalon.net>; Xu, Qian Q
> <qian.q.xu@intel.com>
> Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>;
> O'Driscoll, Tim <tim.odriscoll@intel.com>
> Subject: RE: [dpdk-ci] script to determine target repo (was DPDK Lab)
> 
> Hi Thomas,
> 
> What's the status about the script which determine target repo for each
> patch set?
> Currently I used the rule which you told me before: "the subject prefix on
> the patches should provide the hint. If the majority of the patches start
> with "net/" then it's for next-net, and similarly for crypto. Patches with
> virtio or vhost go to that tree, etc".
> But I'm not sure about the 100% correct repo.
> Looking forward to your script.
> 
> Another thing, Qian told me that you want to know the apply sequence about
> many patches in one patch set.
> I don't apply patches with the order of patchwork id, but the names' order
> about them.
> Such as, patchset 25081-25085, the order of patchwork id is not the
> correct patches' order.
> http://dpdk.org/patch/25081, subject: [dpdk-dev,v2,1/5] net/qede:
> refactoring vport handling code http://dpdk.org/patch/25083, subject:
> [dpdk-dev,v2,2/5] net/qede: refactoring multi-queue implementation
> http://dpdk.org/patch/25084, subject: [dpdk-dev,v2,3/5] net/qede/base:
> upgrade the FW to 8.20.0.0 http://dpdk.org/patch/25082, subject: [dpdk-
> dev,v2,4/5] net/qede: fix VXLAN tunnel Tx offload flag setting
> http://dpdk.org/patch/25085, subject: [dpdk-dev,v2,5/5] net/qede: refactor
> Tx routine After download them, the patches' names are as below:
> http://dpdk.org/patch/25081: dpdk-dev-v2-1-5-net-qede-refactoring-vport-
> handling-code.patch
> http://dpdk.org/patch/25083: dpdk-dev-v2-2-5-net-qede-refactoring-multi-
> queue-implementation.patch
> http://dpdk.org/patch/25084: dpdk-dev-v2-3-5-net-qede-base-upgrade-the-FW-
> to-8.20.0.0.patch
> http://dpdk.org/patch/25082: dpdk-dev-v2-4-5-net-qede-fix-VXLAN-tunnel-Tx-
> offload-flag-setting.patch
> http://dpdk.org/patch/25085: dpdk-dev-v2-5-5-net-qede-refactor-Tx-
> routine.patch
> 
> So applying these patches based on the order of their names, it will be
> correct.
> 
> 
> Best Regards
> Fangfang Wei
> 
> 
> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> Sent: Tuesday, April 25, 2017 2:51 PM
> To: Xu, Qian Q <qian.q.xu@intel.com>
> Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; Wei,
> FangfangX <fangfangx.wei@intel.com>; O'Driscoll, Tim
> <tim.odriscoll@intel.com>; Liu, Yuanhan <yuanhan.liu@intel.com>
> Subject: Re: [dpdk-ci] script to determine target repo (was DPDK Lab)
> 
> 25/04/2017 08:23, Xu, Qian Q:
> > Thanks Thomas and Bruce for your comments. Currently we are trying to
> > start the per patch set performance test investigation and need to know
> the Target repo for each patch set.
> > The goal is to get 100% correct repo to go for each patch set. It's
> important for the credit of the performance result.
> > @Thomas, could you provide any script to support the auto-filter after
> the 17.05 release? Currently, we don't have the ability to make it.
> 
> OK, it is on my todo list.
> 
> Other task: I want to check the series support in the patchwork git.

Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

[Thomas Monjalon]

I agree with Bruce.

Sorry for not having written the scripts yet.
Someone else in Mellanox should do it in July.
In the meantime, do not hesitate to share your code if it speed up things.

Thanks a lot


21/06/2017 10:20, Richardson, Bruce:
> Hi Fangfang,
> 
> My comments on the questions you asked:
> 
> 1. For determining the repo tree to target, I don't believe that we can ever come up with a 100% accurate rule, as the tree to which a set is to be applied can be difficult to determine, so it may be done on the basis of on-list discussion. A 90% accurate rule it what we may have to accept. However, since applying a patchset to a tree should not be a time-consuming operation, I suggest any script produce a list of possible trees in priority order to try. If not net, then try main, etc. etc.
> 
> 2. Using the order from patch titles is correct.
> 
> /Bruce
> 
> > -----Original Message-----
> > From: Wei, FangfangX
> > Sent: Wednesday, June 21, 2017 8:46 AM
> > To: Thomas Monjalon <thomas@monjalon.net>; Xu, Qian Q
> > <qian.q.xu@intel.com>
> > Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>;
> > O'Driscoll, Tim <tim.odriscoll@intel.com>
> > Subject: RE: [dpdk-ci] script to determine target repo (was DPDK Lab)
> > 
> > Hi Thomas,
> > 
> > What's the status about the script which determine target repo for each
> > patch set?
> > Currently I used the rule which you told me before: "the subject prefix on
> > the patches should provide the hint. If the majority of the patches start
> > with "net/" then it's for next-net, and similarly for crypto. Patches with
> > virtio or vhost go to that tree, etc".
> > But I'm not sure about the 100% correct repo.
> > Looking forward to your script.
> > 
> > Another thing, Qian told me that you want to know the apply sequence about
> > many patches in one patch set.
> > I don't apply patches with the order of patchwork id, but the names' order
> > about them.
> > Such as, patchset 25081-25085, the order of patchwork id is not the
> > correct patches' order.
> > http://dpdk.org/patch/25081, subject: [dpdk-dev,v2,1/5] net/qede:
> > refactoring vport handling code http://dpdk.org/patch/25083, subject:
> > [dpdk-dev,v2,2/5] net/qede: refactoring multi-queue implementation
> > http://dpdk.org/patch/25084, subject: [dpdk-dev,v2,3/5] net/qede/base:
> > upgrade the FW to 8.20.0.0 http://dpdk.org/patch/25082, subject: [dpdk-
> > dev,v2,4/5] net/qede: fix VXLAN tunnel Tx offload flag setting
> > http://dpdk.org/patch/25085, subject: [dpdk-dev,v2,5/5] net/qede: refactor
> > Tx routine After download them, the patches' names are as below:
> > http://dpdk.org/patch/25081: dpdk-dev-v2-1-5-net-qede-refactoring-vport-
> > handling-code.patch
> > http://dpdk.org/patch/25083: dpdk-dev-v2-2-5-net-qede-refactoring-multi-
> > queue-implementation.patch
> > http://dpdk.org/patch/25084: dpdk-dev-v2-3-5-net-qede-base-upgrade-the-FW-
> > to-8.20.0.0.patch
> > http://dpdk.org/patch/25082: dpdk-dev-v2-4-5-net-qede-fix-VXLAN-tunnel-Tx-
> > offload-flag-setting.patch
> > http://dpdk.org/patch/25085: dpdk-dev-v2-5-5-net-qede-refactor-Tx-
> > routine.patch
> > 
> > So applying these patches based on the order of their names, it will be
> > correct.
> > 
> > 
> > Best Regards
> > Fangfang Wei
> > 
> > 
> > -----Original Message-----
> > From: Thomas Monjalon [mailto:thomas@monjalon.net]
> > Sent: Tuesday, April 25, 2017 2:51 PM
> > To: Xu, Qian Q <qian.q.xu@intel.com>
> > Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; Wei,
> > FangfangX <fangfangx.wei@intel.com>; O'Driscoll, Tim
> > <tim.odriscoll@intel.com>; Liu, Yuanhan <yuanhan.liu@intel.com>
> > Subject: Re: [dpdk-ci] script to determine target repo (was DPDK Lab)
> > 
> > 25/04/2017 08:23, Xu, Qian Q:
> > > Thanks Thomas and Bruce for your comments. Currently we are trying to
> > > start the per patch set performance test investigation and need to know
> > the Target repo for each patch set.
> > > The goal is to get 100% correct repo to go for each patch set. It's
> > important for the credit of the performance result.
> > > @Thomas, could you provide any script to support the auto-filter after
> > the 17.05 release? Currently, we don't have the ability to make it.
> > 
> > OK, it is on my todo list.
> > 
> > Other task: I want to check the series support in the patchwork git.

Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

[Xu, Qian Q]

Thomas/Bruce
1. For determining the repo tree to target, I don't believe that we can ever
> come up with a 100% accurate rule, as the tree to which a set is to be applied
> can be difficult to determine, so it may be done on the basis of on-list discussion.
> A 90% accurate rule it what we may have to accept.

-- Then if we find the performance issue, then maybe it's a false alarm due to apply to the wrong repo. So, we may face many false alarms according with the time. 
Then people may not treat the performance issue as a problem, so I still think we need to try 100% accurate to have a more trustable result when we send out the alarm. 


> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> Sent: Wednesday, June 21, 2017 4:33 PM
> To: Wei, FangfangX <fangfangx.wei@intel.com>
> Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; Xu, Qian Q
> <qian.q.xu@intel.com>; O'Driscoll, Tim <tim.odriscoll@intel.com>; Eugene
> Voronov <eugene@mellanox.com>
> Subject: Re: [dpdk-ci] script to determine target repo (was DPDK Lab)
> 
> I agree with Bruce.
> 
> Sorry for not having written the scripts yet.
> Someone else in Mellanox should do it in July.
> In the meantime, do not hesitate to share your code if it speed up things.
> 
> Thanks a lot
> 
> 
> 21/06/2017 10:20, Richardson, Bruce:
> > Hi Fangfang,
> >
> > My comments on the questions you asked:
> >
> > 1. For determining the repo tree to target, I don't believe that we can ever
> come up with a 100% accurate rule, as the tree to which a set is to be applied
> can be difficult to determine, so it may be done on the basis of on-list discussion.
> A 90% accurate rule it what we may have to accept. However, since applying a
> patchset to a tree should not be a time-consuming operation, I suggest any
> script produce a list of possible trees in priority order to try. If not net, then try
> main, etc. etc.
> >
> > 2. Using the order from patch titles is correct.
> >
> > /Bruce

Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

[Richardson, Bruce]

> -----Original Message-----
> From: Xu, Qian Q
> Sent: Friday, June 23, 2017 9:44 AM
> To: Thomas Monjalon <thomas@monjalon.net>; Wei, FangfangX
> <fangfangx.wei@intel.com>
> Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>;
> O'Driscoll, Tim <tim.odriscoll@intel.com>; Eugene Voronov
> <eugene@mellanox.com>
> Subject: RE: [dpdk-ci] script to determine target repo (was DPDK Lab)
> 
> Thomas/Bruce
> 1. For determining the repo tree to target, I don't believe that we can
> ever
> > come up with a 100% accurate rule, as the tree to which a set is to be
> > applied can be difficult to determine, so it may be done on the basis of
> on-list discussion.
> > A 90% accurate rule it what we may have to accept.
> 
> -- Then if we find the performance issue, then maybe it's a false alarm
> due to apply to the wrong repo. So, we may face many false alarms
> according with the time.
> Then people may not treat the performance issue as a problem, so I still
> think we need to try 100% accurate to have a more trustable result when we
> send out the alarm.

I find that rather improbable, and not worth considering. For that to per a problem multiple unlikely events have to occur:
1) we mis-identify the tree on which the set is to be applied (we should be able to get to 90% accuracy here)
2) the patchset must apply cleanly to the "wrong" tree (this is reasonably likely, but it's still another condition that has to be met for us to have a problem)
3) the patchset has to cause a performance regression in the "wrong" tree
4) but NOT cause a regression when in the right tree.

If we assume 90% accuracy of tree identification, optimistically that 90% of patches will apply to the wrong tree, that 5% of patches cause a performance regression (an overestimate IMHO), and that even 1/3 of those won't cause a performance regression in the right tree (a very overestimate IMHO, I would expect just about none of them to even have this), it still means that only about 1 patch in 1000 will show as a false positive performance regression.

0.1 (mis-identify) * 0.9 (applies ok) * 0.05 (regression) * 0.33 (no regression) = 0.0015, or 0.15%

So worst case, I still don't think we have a problem for the scenario you describe.

/Bruce

> 
> 
> > -----Original Message-----
> > From: Thomas Monjalon [mailto:thomas@monjalon.net]
> > Sent: Wednesday, June 21, 2017 4:33 PM
> > To: Wei, FangfangX <fangfangx.wei@intel.com>
> > Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; Xu,
> > Qian Q <qian.q.xu@intel.com>; O'Driscoll, Tim
> > <tim.odriscoll@intel.com>; Eugene Voronov <eugene@mellanox.com>
> > Subject: Re: [dpdk-ci] script to determine target repo (was DPDK Lab)
> >
> > I agree with Bruce.
> >
> > Sorry for not having written the scripts yet.
> > Someone else in Mellanox should do it in July.
> > In the meantime, do not hesitate to share your code if it speed up
> things.
> >
> > Thanks a lot
> >
> >
> > 21/06/2017 10:20, Richardson, Bruce:
> > > Hi Fangfang,
> > >
> > > My comments on the questions you asked:
> > >
> > > 1. For determining the repo tree to target, I don't believe that we
> > > can ever
> > come up with a 100% accurate rule, as the tree to which a set is to be
> > applied can be difficult to determine, so it may be done on the basis of
> on-list discussion.
> > A 90% accurate rule it what we may have to accept. However, since
> > applying a patchset to a tree should not be a time-consuming
> > operation, I suggest any script produce a list of possible trees in
> > priority order to try. If not net, then try main, etc. etc.
> > >
> > > 2. Using the order from patch titles is correct.
> > >
> > > /Bruce

Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

[Xu, Qian Q]

> -----Original Message-----
> From: Richardson, Bruce
> Sent: Friday, June 23, 2017 5:30 PM
> To: Xu, Qian Q <qian.q.xu@intel.com>; Thomas Monjalon
> <thomas@monjalon.net>; Wei, FangfangX <fangfangx.wei@intel.com>
> Cc: ci@dpdk.org; O'Driscoll, Tim <tim.odriscoll@intel.com>; Eugene Voronov
> <eugene@mellanox.com>
> Subject: RE: [dpdk-ci] script to determine target repo (was DPDK Lab)
> 
> 
> 
> > -----Original Message-----
> > From: Xu, Qian Q
> > Sent: Friday, June 23, 2017 9:44 AM
> > To: Thomas Monjalon <thomas@monjalon.net>; Wei, FangfangX
> > <fangfangx.wei@intel.com>
> > Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>;
> > O'Driscoll, Tim <tim.odriscoll@intel.com>; Eugene Voronov
> > <eugene@mellanox.com>
> > Subject: RE: [dpdk-ci] script to determine target repo (was DPDK Lab)
> >
> > Thomas/Bruce
> > 1. For determining the repo tree to target, I don't believe that we
> > can ever
> > > come up with a 100% accurate rule, as the tree to which a set is to
> > > be applied can be difficult to determine, so it may be done on the
> > > basis of
> > on-list discussion.
> > > A 90% accurate rule it what we may have to accept.
> >
> > -- Then if we find the performance issue, then maybe it's a false
> > alarm due to apply to the wrong repo. So, we may face many false
> > alarms according with the time.
> > Then people may not treat the performance issue as a problem, so I
> > still think we need to try 100% accurate to have a more trustable
> > result when we send out the alarm.
> 
> I find that rather improbable, and not worth considering. For that to per a
> problem multiple unlikely events have to occur:
> 1) we mis-identify the tree on which the set is to be applied (we should be able
> to get to 90% accuracy here)
> 2) the patchset must apply cleanly to the "wrong" tree (this is reasonably likely,
> but it's still another condition that has to be met for us to have a problem)
> 3) the patchset has to cause a performance regression in the "wrong" tree
> 4) but NOT cause a regression when in the right tree.
> 
> If we assume 90% accuracy of tree identification, optimistically that 90% of
> patches will apply to the wrong tree, that 5% of patches cause a performance
> regression (an overestimate IMHO), and that even 1/3 of those won't cause a
> performance regression in the right tree (a very overestimate IMHO, I would
> expect just about none of them to even have this), it still means that only about
> 1 patch in 1000 will show as a false positive performance regression.
> 
> 0.1 (mis-identify) * 0.9 (applies ok) * 0.05 (regression) * 0.33 (no regression) =
> 0.0015, or 0.15%
> 
> So worst case, I still don't think we have a problem for the scenario you describe.

OK, Bruce, so the question is that how can we ensure 90% accuracy? How to check if it's 90% or 80%? 

Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

[Jerin Jacob]

-----Original Message-----
> Date: Fri, 23 Jun 2017 09:30:28 +0000
> From: "Richardson, Bruce" <bruce.richardson@intel.com>
> To: "Xu, Qian Q" <qian.q.xu@intel.com>, Thomas Monjalon
>  <thomas@monjalon.net>, "Wei, FangfangX" <fangfangx.wei@intel.com>
> CC: "ci@dpdk.org" <ci@dpdk.org>, "O'Driscoll, Tim"
>  <tim.odriscoll@intel.com>, Eugene Voronov <eugene@mellanox.com>
> Subject: Re: [dpdk-ci] script to determine target repo (was DPDK Lab)
> 
> 
> 
> > -----Original Message-----
> > From: Xu, Qian Q
> > Sent: Friday, June 23, 2017 9:44 AM
> > To: Thomas Monjalon <thomas@monjalon.net>; Wei, FangfangX
> > <fangfangx.wei@intel.com>
> > Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>;
> > O'Driscoll, Tim <tim.odriscoll@intel.com>; Eugene Voronov
> > <eugene@mellanox.com>
> > Subject: RE: [dpdk-ci] script to determine target repo (was DPDK Lab)
> > 
> > Thomas/Bruce
> > 1. For determining the repo tree to target, I don't believe that we can
> > ever
> > > come up with a 100% accurate rule, as the tree to which a set is to be
> > > applied can be difficult to determine, so it may be done on the basis of
> > on-list discussion.
> > > A 90% accurate rule it what we may have to accept.
> > 
> > -- Then if we find the performance issue, then maybe it's a false alarm
> > due to apply to the wrong repo. So, we may face many false alarms
> > according with the time.
> > Then people may not treat the performance issue as a problem, so I still
> > think we need to try 100% accurate to have a more trustable result when we
> > send out the alarm.
> 
> I find that rather improbable, and not worth considering. For that to per a problem multiple unlikely events have to occur:
> 1) we mis-identify the tree on which the set is to be applied (we should be able to get to 90% accuracy here)
> 2) the patchset must apply cleanly to the "wrong" tree (this is reasonably likely, but it's still another condition that has to be met for us to have a problem)
> 3) the patchset has to cause a performance regression in the "wrong" tree
> 4) but NOT cause a regression when in the right tree.
> 
> If we assume 90% accuracy of tree identification, optimistically that 90% of patches will apply to the wrong tree, that 5% of patches cause a performance regression (an overestimate IMHO), and that even 1/3 of those won't cause a performance regression in the right tree (a very overestimate IMHO, I would expect just about none of them to even have this), it still means that only about 1 patch in 1000 will show as a false positive performance regression.
> 
> 0.1 (mis-identify) * 0.9 (applies ok) * 0.05 (regression) * 0.33 (no regression) = 0.0015, or 0.15%
> 
> So worst case, I still don't think we have a problem for the scenario you describe.

Another option could be to fix the problem at source. Where the targeted
script can run at the source, identify the target repo and embedded that
information in 1st patch in series. I guess, that will provide a means
(user can change the target repo at source if required)
to fixup the remaining 10% accuracy.



> > 
> > 
> > > -----Original Message-----
> > > From: Thomas Monjalon [mailto:thomas@monjalon.net]
> > > Sent: Wednesday, June 21, 2017 4:33 PM
> > > To: Wei, FangfangX <fangfangx.wei@intel.com>
> > > Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; Xu,
> > > Qian Q <qian.q.xu@intel.com>; O'Driscoll, Tim
> > > <tim.odriscoll@intel.com>; Eugene Voronov <eugene@mellanox.com>
> > > Subject: Re: [dpdk-ci] script to determine target repo (was DPDK Lab)
> > >
> > > I agree with Bruce.
> > >
> > > Sorry for not having written the scripts yet.
> > > Someone else in Mellanox should do it in July.
> > > In the meantime, do not hesitate to share your code if it speed up
> > things.
> > >
> > > Thanks a lot
> > >
> > >
> > > 21/06/2017 10:20, Richardson, Bruce:
> > > > Hi Fangfang,
> > > >
> > > > My comments on the questions you asked:
> > > >
> > > > 1. For determining the repo tree to target, I don't believe that we
> > > > can ever
> > > come up with a 100% accurate rule, as the tree to which a set is to be
> > > applied can be difficult to determine, so it may be done on the basis of
> > on-list discussion.
> > > A 90% accurate rule it what we may have to accept. However, since
> > > applying a patchset to a tree should not be a time-consuming
> > > operation, I suggest any script produce a list of possible trees in
> > > priority order to try. If not net, then try main, etc. etc.
> > > >
> > > > 2. Using the order from patch titles is correct.
> > > >
> > > > /Bruce

Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

[Xu, Qian Q]

yes，that's the way I want, every submitter should know their target repo

发自我的 iPhone

> 在 2017年6月26日，下午2:24，Jerin Jacob <jerin.jacob@caviumnetworks.com> 写道：
> 
> -----Original Message-----
>> Date: Fri, 23 Jun 2017 09:30:28 +0000
>> From: "Richardson, Bruce" <bruce.richardson@intel.com>
>> To: "Xu, Qian Q" <qian.q.xu@intel.com>, Thomas Monjalon
>> <thomas@monjalon.net>, "Wei, FangfangX" <fangfangx.wei@intel.com>
>> CC: "ci@dpdk.org" <ci@dpdk.org>, "O'Driscoll, Tim"
>> <tim.odriscoll@intel.com>, Eugene Voronov <eugene@mellanox.com>
>> Subject: Re: [dpdk-ci] script to determine target repo (was DPDK Lab)
>> 
>> 
>> 
>>> -----Original Message-----
>>> From: Xu, Qian Q
>>> Sent: Friday, June 23, 2017 9:44 AM
>>> To: Thomas Monjalon <thomas@monjalon.net>; Wei, FangfangX
>>> <fangfangx.wei@intel.com>
>>> Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>;
>>> O'Driscoll, Tim <tim.odriscoll@intel.com>; Eugene Voronov
>>> <eugene@mellanox.com>
>>> Subject: RE: [dpdk-ci] script to determine target repo (was DPDK Lab)
>>> 
>>> Thomas/Bruce
>>> 1. For determining the repo tree to target, I don't believe that we can
>>> ever
>>>> come up with a 100% accurate rule, as the tree to which a set is to be
>>>> applied can be difficult to determine, so it may be done on the basis of
>>> on-list discussion.
>>>> A 90% accurate rule it what we may have to accept.
>>> 
>>> -- Then if we find the performance issue, then maybe it's a false alarm
>>> due to apply to the wrong repo. So, we may face many false alarms
>>> according with the time.
>>> Then people may not treat the performance issue as a problem, so I still
>>> think we need to try 100% accurate to have a more trustable result when we
>>> send out the alarm.
>> 
>> I find that rather improbable, and not worth considering. For that to per a problem multiple unlikely events have to occur:
>> 1) we mis-identify the tree on which the set is to be applied (we should be able to get to 90% accuracy here)
>> 2) the patchset must apply cleanly to the "wrong" tree (this is reasonably likely, but it's still another condition that has to be met for us to have a problem)
>> 3) the patchset has to cause a performance regression in the "wrong" tree
>> 4) but NOT cause a regression when in the right tree.
>> 
>> If we assume 90% accuracy of tree identification, optimistically that 90% of patches will apply to the wrong tree, that 5% of patches cause a performance regression (an overestimate IMHO), and that even 1/3 of those won't cause a performance regression in the right tree (a very overestimate IMHO, I would expect just about none of them to even have this), it still means that only about 1 patch in 1000 will show as a false positive performance regression.
>> 
>> 0.1 (mis-identify) * 0.9 (applies ok) * 0.05 (regression) * 0.33 (no regression) = 0.0015, or 0.15%
>> 
>> So worst case, I still don't think we have a problem for the scenario you describe.
> 
> Another option could be to fix the problem at source. Where the targeted
> script can run at the source, identify the target repo and embedded that
> information in 1st patch in series. I guess, that will provide a means
> (user can change the target repo at source if required)
> to fixup the remaining 10% accuracy.
> 
> 
> 
>>> 
>>> 
>>>> -----Original Message-----
>>>> From: Thomas Monjalon [mailto:thomas@monjalon.net]
>>>> Sent: Wednesday, June 21, 2017 4:33 PM
>>>> To: Wei, FangfangX <fangfangx.wei@intel.com>
>>>> Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; Xu,
>>>> Qian Q <qian.q.xu@intel.com>; O'Driscoll, Tim
>>>> <tim.odriscoll@intel.com>; Eugene Voronov <eugene@mellanox.com>
>>>> Subject: Re: [dpdk-ci] script to determine target repo (was DPDK Lab)
>>>> 
>>>> I agree with Bruce.
>>>> 
>>>> Sorry for not having written the scripts yet.
>>>> Someone else in Mellanox should do it in July.
>>>> In the meantime, do not hesitate to share your code if it speed up
>>> things.
>>>> 
>>>> Thanks a lot
>>>> 
>>>> 
>>>> 21/06/2017 10:20, Richardson, Bruce:
>>>>> Hi Fangfang,
>>>>> 
>>>>> My comments on the questions you asked:
>>>>> 
>>>>> 1. For determining the repo tree to target, I don't believe that we
>>>>> can ever
>>>> come up with a 100% accurate rule, as the tree to which a set is to be
>>>> applied can be difficult to determine, so it may be done on the basis of
>>> on-list discussion.
>>>> A 90% accurate rule it what we may have to accept. However, since
>>>> applying a patchset to a tree should not be a time-consuming
>>>> operation, I suggest any script produce a list of possible trees in
>>>> priority order to try. If not net, then try main, etc. etc.
>>>>> 
>>>>> 2. Using the order from patch titles is correct.
>>>>> 
>>>>> /Bruce

Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

[Wei, FangfangX]

Hi Thomas,
I've write a script which can download patchset(not single patch) from dpdk.org. I'm preparing to upload it into dpdk-ci repo.
Which director can it be put in?
And how can I upload the script into dpdk.org? For I haven't uploaded a script into dpdk.org before.

BTW, this script is a part of per patchset performance testing.

Best Regards
Fangfang Wei

-----Original Message-----
From: Thomas Monjalon [mailto:thomas@monjalon.net] 
Sent: Wednesday, June 21, 2017 4:33 PM
To: Wei, FangfangX <fangfangx.wei@intel.com>
Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; Xu, Qian Q <qian.q.xu@intel.com>; O'Driscoll, Tim <tim.odriscoll@intel.com>; Eugene Voronov <eugene@mellanox.com>
Subject: Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

I agree with Bruce.

Sorry for not having written the scripts yet.
Someone else in Mellanox should do it in July.
In the meantime, do not hesitate to share your code if it speed up things.

Thanks a lot


21/06/2017 10:20, Richardson, Bruce:
> Hi Fangfang,
> 
> My comments on the questions you asked:
> 
> 1. For determining the repo tree to target, I don't believe that we can ever come up with a 100% accurate rule, as the tree to which a set is to be applied can be difficult to determine, so it may be done on the basis of on-list discussion. A 90% accurate rule it what we may have to accept. However, since applying a patchset to a tree should not be a time-consuming operation, I suggest any script produce a list of possible trees in priority order to try. If not net, then try main, etc. etc.
> 
> 2. Using the order from patch titles is correct.
> 
> /Bruce
> 
> > -----Original Message-----
> > From: Wei, FangfangX
> > Sent: Wednesday, June 21, 2017 8:46 AM
> > To: Thomas Monjalon <thomas@monjalon.net>; Xu, Qian Q 
> > <qian.q.xu@intel.com>
> > Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; 
> > O'Driscoll, Tim <tim.odriscoll@intel.com>
> > Subject: RE: [dpdk-ci] script to determine target repo (was DPDK 
> > Lab)
> > 
> > Hi Thomas,
> > 
> > What's the status about the script which determine target repo for 
> > each patch set?
> > Currently I used the rule which you told me before: "the subject 
> > prefix on the patches should provide the hint. If the majority of 
> > the patches start with "net/" then it's for next-net, and similarly 
> > for crypto. Patches with virtio or vhost go to that tree, etc".
> > But I'm not sure about the 100% correct repo.
> > Looking forward to your script.
> > 
> > Another thing, Qian told me that you want to know the apply sequence 
> > about many patches in one patch set.
> > I don't apply patches with the order of patchwork id, but the names' 
> > order about them.
> > Such as, patchset 25081-25085, the order of patchwork id is not the 
> > correct patches' order.
> > http://dpdk.org/patch/25081, subject: [dpdk-dev,v2,1/5] net/qede:
> > refactoring vport handling code http://dpdk.org/patch/25083, subject:
> > [dpdk-dev,v2,2/5] net/qede: refactoring multi-queue implementation 
> > http://dpdk.org/patch/25084, subject: [dpdk-dev,v2,3/5] net/qede/base:
> > upgrade the FW to 8.20.0.0 http://dpdk.org/patch/25082, subject: 
> > [dpdk- dev,v2,4/5] net/qede: fix VXLAN tunnel Tx offload flag 
> > setting http://dpdk.org/patch/25085, subject: [dpdk-dev,v2,5/5] 
> > net/qede: refactor Tx routine After download them, the patches' names are as below:
> > http://dpdk.org/patch/25081: 
> > dpdk-dev-v2-1-5-net-qede-refactoring-vport-
> > handling-code.patch
> > http://dpdk.org/patch/25083: 
> > dpdk-dev-v2-2-5-net-qede-refactoring-multi-
> > queue-implementation.patch
> > http://dpdk.org/patch/25084: 
> > dpdk-dev-v2-3-5-net-qede-base-upgrade-the-FW-
> > to-8.20.0.0.patch
> > http://dpdk.org/patch/25082: 
> > dpdk-dev-v2-4-5-net-qede-fix-VXLAN-tunnel-Tx-
> > offload-flag-setting.patch
> > http://dpdk.org/patch/25085: dpdk-dev-v2-5-5-net-qede-refactor-Tx-
> > routine.patch
> > 
> > So applying these patches based on the order of their names, it will 
> > be correct.
> > 
> > 
> > Best Regards
> > Fangfang Wei
> > 
> > 
> > -----Original Message-----
> > From: Thomas Monjalon [mailto:thomas@monjalon.net]
> > Sent: Tuesday, April 25, 2017 2:51 PM
> > To: Xu, Qian Q <qian.q.xu@intel.com>
> > Cc: ci@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; 
> > Wei, FangfangX <fangfangx.wei@intel.com>; O'Driscoll, Tim 
> > <tim.odriscoll@intel.com>; Liu, Yuanhan <yuanhan.liu@intel.com>
> > Subject: Re: [dpdk-ci] script to determine target repo (was DPDK 
> > Lab)
> > 
> > 25/04/2017 08:23, Xu, Qian Q:
> > > Thanks Thomas and Bruce for your comments. Currently we are trying 
> > > to start the per patch set performance test investigation and need 
> > > to know
> > the Target repo for each patch set.
> > > The goal is to get 100% correct repo to go for each patch set. 
> > > It's
> > important for the credit of the performance result.
> > > @Thomas, could you provide any script to support the auto-filter 
> > > after
> > the 17.05 release? Currently, we don't have the ability to make it.
> > 
> > OK, it is on my todo list.
> > 
> > Other task: I want to check the series support in the patchwork git.

Re: [dpdk-ci] script to determine target repo (was DPDK Lab)

[Thomas Monjalon]

Hi,

30/06/2017 10:41, Wei, FangfangX:
> Hi Thomas,
> I've write a script which can download patchset(not single patch) from dpdk.org. I'm preparing to upload it into dpdk-ci repo.
> Which director can it be put in?
> And how can I upload the script into dpdk.org? For I haven't uploaded a script into dpdk.org before.

Please send it as a patch with git-send-email to ci@dpdk.org.
Then we will be able to review it before integrating it to the repo.
Thanks