BRCM 57414 Performance Test

BRCM 57414 Performance Test

[Patrick Robb]

Hi Ajit,

One of the tests we run for the BRCM 57414 NIC in the dpdk community
lab is a single core forwarding test in which we try to match line
rate on the NIC, and protect against any performance regressions in
DPDK. We track the MPPS forwarded between interfaces on the DUT,
compare that metric against the most recent "baseline" run, and if the
delta is more than 5% it is a fail. So the idea is if a significant
regression is introduced on a patch, CI testing catches that.

Right now on one of our ARM systems with brcm57414 this test is having
high variance... more than the 5% threshold in some cases. I think
this may in some way relate to the maintenance we did a few weeks back
- we were seeing like .2% variance before. The expected throughput
seems consistent, but the results have higher variance than normal.
Obviously something is slightly wrong, and the system needs to be
re-tuned a little or something. We can look at it (prefer to do any
maintenance once all RCs are complete), but in the interim I want to
just bump the accepted Delta from 5% to 7%, just to stop the false
fails, but still maintain some coverage. Once we tune the system and
reduce the results variance, we can return to 5%.

Does this sound fine to you?

Thanks,
Patrick

-- 
Patrick Robb

Technical Service Manager

UNH InterOperability Laboratory

21 Madbury Rd, Suite 100, Durham, NH 03824

www.iol.unh.edu

Re: BRCM 57414 Performance Test

[Ajit Khaparde]

[-- Attachment #1: Type: text/plain, Size: 1694 bytes --]

Hi Patrick,
Yes, it makes sense.
Thanks for the note. Please let me know once its sorted out.

Thanks
Ajit

On Thu, Mar 14, 2024 at 7:46 AM Patrick Robb <probb@iol.unh.edu> wrote:
>
> Hi Ajit,
>
> One of the tests we run for the BRCM 57414 NIC in the dpdk community
> lab is a single core forwarding test in which we try to match line
> rate on the NIC, and protect against any performance regressions in
> DPDK. We track the MPPS forwarded between interfaces on the DUT,
> compare that metric against the most recent "baseline" run, and if the
> delta is more than 5% it is a fail. So the idea is if a significant
> regression is introduced on a patch, CI testing catches that.
>
> Right now on one of our ARM systems with brcm57414 this test is having
> high variance... more than the 5% threshold in some cases. I think
> this may in some way relate to the maintenance we did a few weeks back
> - we were seeing like .2% variance before. The expected throughput
> seems consistent, but the results have higher variance than normal.
> Obviously something is slightly wrong, and the system needs to be
> re-tuned a little or something. We can look at it (prefer to do any
> maintenance once all RCs are complete), but in the interim I want to
> just bump the accepted Delta from 5% to 7%, just to stop the false
> fails, but still maintain some coverage. Once we tune the system and
> reduce the results variance, we can return to 5%.
>
> Does this sound fine to you?
>
> Thanks,
> Patrick
>
> --
> Patrick Robb
>
> Technical Service Manager
>
> UNH InterOperability Laboratory
>
> 21 Madbury Rd, Suite 100, Durham, NH 03824
>
> www.iol.unh.edu

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4218 bytes --]