[dts] [PATCH v1] tests/vf_packet_rxtx: fix test case failures in vf_reset

[dts] [PATCH v1] tests/vf_packet_rxtx: fix test case failures in vf_reset

[Herbert Guan]

1) Clear port stats before send/recv test packets
2) Ensure igb_uio driver is installed if driver='igb_uio'


Signed-off-by: Herbert Guan <herbert.guan@arm.com>
---
 tests/TestSuite_vf_packet_rxtx.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tests/TestSuite_vf_packet_rxtx.py b/tests/TestSuite_vf_packet_rxtx.py
index 63f4820..f0b7721 100644
--- a/tests/TestSuite_vf_packet_rxtx.py
+++ b/tests/TestSuite_vf_packet_rxtx.py
@@ -125,6 +125,9 @@ class TestVfPacketRxtx(TestCase):
     def packet_rx_tx(self, driver='default'):
 
         if driver == 'igb_uio':
+            # ensure igb_uio driver is installed
+            self.dut.send_expect('(lsmod | grep uio) || (modprobe uio)', '#')
+            self.dut.send_expect('(lsmod | grep igb_uio) || (insmod ./%s/kmod/igb_uio.ko)' % self.target,'#')
             self.setup_2pf_2vf_1vm_env(driver='igb_uio')
         else:
             self.setup_2pf_2vf_1vm_env(driver='')
@@ -279,6 +282,7 @@ class TestVfPacketRxtx(TestCase):
         rx_port = tx_port
 
         dst_mac = pmd0_vf0_mac
+        self.vm0_testpmd.execute_cmd('clear port stats all')
         self.tester.sendpkt_bg(tx_port, dst_mac)
 
         #vf port stop/start can trigger reset action
-- 
1.8.3.1

Re: [dts] [PATCH v1] tests/vf_packet_rxtx: fix test case failures in vf_reset

[Liu, Yong]

Hi Herbert,
Why required driver is not insmod be previous setup?
As one principle, we want to limit the influence of environment change in the suite.

Thanks,
Marvin

> -----Original Message-----
> From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Herbert Guan
> Sent: Wednesday, February 14, 2018 2:29 PM
> To: dts@dpdk.org
> Cc: Herbert Guan <herbert.guan@arm.com>
> Subject: [dts] [PATCH v1] tests/vf_packet_rxtx: fix test case failures in
> vf_reset
> 
> 1) Clear port stats before send/recv test packets
> 2) Ensure igb_uio driver is installed if driver='igb_uio'
> 
> 
> Signed-off-by: Herbert Guan <herbert.guan@arm.com>
> ---
>  tests/TestSuite_vf_packet_rxtx.py | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/tests/TestSuite_vf_packet_rxtx.py
> b/tests/TestSuite_vf_packet_rxtx.py
> index 63f4820..f0b7721 100644
> --- a/tests/TestSuite_vf_packet_rxtx.py
> +++ b/tests/TestSuite_vf_packet_rxtx.py
> @@ -125,6 +125,9 @@ class TestVfPacketRxtx(TestCase):
>      def packet_rx_tx(self, driver='default'):
> 
>          if driver == 'igb_uio':
> +            # ensure igb_uio driver is installed
> +            self.dut.send_expect('(lsmod | grep uio) || (modprobe uio)',
> '#')
> +            self.dut.send_expect('(lsmod | grep igb_uio) ||
> (insmod ./%s/kmod/igb_uio.ko)' % self.target,'#')
>              self.setup_2pf_2vf_1vm_env(driver='igb_uio')
>          else:
>              self.setup_2pf_2vf_1vm_env(driver='')
> @@ -279,6 +282,7 @@ class TestVfPacketRxtx(TestCase):
>          rx_port = tx_port
> 
>          dst_mac = pmd0_vf0_mac
> +        self.vm0_testpmd.execute_cmd('clear port stats all')
>          self.tester.sendpkt_bg(tx_port, dst_mac)
> 
>          #vf port stop/start can trigger reset action
> --
> 1.8.3.1

Re: [dts] [PATCH v1] tests/vf_packet_rxtx: fix test case failures in vf_reset

[Herbert Guan]

Marvin,

When the execution config file selects driver mode to a certain mode (vfio, igb_uio, etc.), then both host and VM NICs will use this mode in case setup.  If some case need VM to use the igb_uio mode, regardless of the driver mode in execution configure file, we need to ensure the required drivers have been installed.  

If we want to restore the environment, we need to add some extra logic to the case script.  If the uio or igb_uio driver is not installed, then install it and make flag.  This script need to remove installed drivers that were not installed before running the test.

Best regards,
Herbert

> -----Original Message-----
> From: Liu, Yong [mailto:yong.liu@intel.com]
> Sent: Friday, March 2, 2018 10:08
> To: Herbert Guan <Herbert.Guan@arm.com>; dts@dpdk.org
> Subject: RE: [dts] [PATCH v1] tests/vf_packet_rxtx: fix test case failures in
> vf_reset
> 
> Hi Herbert,
> Why required driver is not insmod be previous setup?
> As one principle, we want to limit the influence of environment change in the
> suite.
> 
> Thanks,
> Marvin
> 
> > -----Original Message-----
> > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Herbert Guan
> > Sent: Wednesday, February 14, 2018 2:29 PM
> > To: dts@dpdk.org
> > Cc: Herbert Guan <herbert.guan@arm.com>
> > Subject: [dts] [PATCH v1] tests/vf_packet_rxtx: fix test case failures
> > in vf_reset
> >
> > 1) Clear port stats before send/recv test packets
> > 2) Ensure igb_uio driver is installed if driver='igb_uio'
> >
> >
> > Signed-off-by: Herbert Guan <herbert.guan@arm.com>
> > ---
> >  tests/TestSuite_vf_packet_rxtx.py | 4 ++++
> >  1 file changed, 4 insertions(+)
> >
> > diff --git a/tests/TestSuite_vf_packet_rxtx.py
> > b/tests/TestSuite_vf_packet_rxtx.py
> > index 63f4820..f0b7721 100644
> > --- a/tests/TestSuite_vf_packet_rxtx.py
> > +++ b/tests/TestSuite_vf_packet_rxtx.py
> > @@ -125,6 +125,9 @@ class TestVfPacketRxtx(TestCase):
> >      def packet_rx_tx(self, driver='default'):
> >
> >          if driver == 'igb_uio':
> > +            # ensure igb_uio driver is installed
> > +            self.dut.send_expect('(lsmod | grep uio) || (modprobe
> > + uio)',
> > '#')
> > +            self.dut.send_expect('(lsmod | grep igb_uio) ||
> > (insmod ./%s/kmod/igb_uio.ko)' % self.target,'#')
> >              self.setup_2pf_2vf_1vm_env(driver='igb_uio')
> >          else:
> >              self.setup_2pf_2vf_1vm_env(driver='')
> > @@ -279,6 +282,7 @@ class TestVfPacketRxtx(TestCase):
> >          rx_port = tx_port
> >
> >          dst_mac = pmd0_vf0_mac
> > +        self.vm0_testpmd.execute_cmd('clear port stats all')
> >          self.tester.sendpkt_bg(tx_port, dst_mac)
> >
> >          #vf port stop/start can trigger reset action
> > --
> > 1.8.3.1

Re: [dts] [PATCH v1] tests/vf_packet_rxtx: fix test case failures in vf_reset

[Liu, Yong]

Herbert,
We have also incurred your problem in some VF related cases. Look like most VF related cases need one default module (igb_uio or vfio noiommu) in VM.
As default module should be bound with VM, I'd like to add one global option for this default module.
Meanwhile test suite can change the default value in their own configuration.

Thanks,
Marvin

> -----Original Message-----
> From: Herbert Guan [mailto:Herbert.Guan@arm.com]
> Sent: Friday, March 02, 2018 2:58 PM
> To: Liu, Yong <yong.liu@intel.com>; dts@dpdk.org
> Cc: nd <nd@arm.com>
> Subject: RE: [dts] [PATCH v1] tests/vf_packet_rxtx: fix test case failures
> in vf_reset
> 
> Marvin,
> 
> When the execution config file selects driver mode to a certain mode (vfio,
> igb_uio, etc.), then both host and VM NICs will use this mode in case
> setup.  If some case need VM to use the igb_uio mode, regardless of the
> driver mode in execution configure file, we need to ensure the required
> drivers have been installed.
> 
> If we want to restore the environment, we need to add some extra logic to
> the case script.  If the uio or igb_uio driver is not installed, then
> install it and make flag.  This script need to remove installed drivers
> that were not installed before running the test.
> 
> Best regards,
> Herbert
> 
> > -----Original Message-----
> > From: Liu, Yong [mailto:yong.liu@intel.com]
> > Sent: Friday, March 2, 2018 10:08
> > To: Herbert Guan <Herbert.Guan@arm.com>; dts@dpdk.org
> > Subject: RE: [dts] [PATCH v1] tests/vf_packet_rxtx: fix test case
> failures in
> > vf_reset
> >
> > Hi Herbert,
> > Why required driver is not insmod be previous setup?
> > As one principle, we want to limit the influence of environment change
> in the
> > suite.
> >
> > Thanks,
> > Marvin
> >
> > > -----Original Message-----
> > > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Herbert Guan
> > > Sent: Wednesday, February 14, 2018 2:29 PM
> > > To: dts@dpdk.org
> > > Cc: Herbert Guan <herbert.guan@arm.com>
> > > Subject: [dts] [PATCH v1] tests/vf_packet_rxtx: fix test case failures
> > > in vf_reset
> > >
> > > 1) Clear port stats before send/recv test packets
> > > 2) Ensure igb_uio driver is installed if driver='igb_uio'
> > >
> > >
> > > Signed-off-by: Herbert Guan <herbert.guan@arm.com>
> > > ---
> > >  tests/TestSuite_vf_packet_rxtx.py | 4 ++++
> > >  1 file changed, 4 insertions(+)
> > >
> > > diff --git a/tests/TestSuite_vf_packet_rxtx.py
> > > b/tests/TestSuite_vf_packet_rxtx.py
> > > index 63f4820..f0b7721 100644
> > > --- a/tests/TestSuite_vf_packet_rxtx.py
> > > +++ b/tests/TestSuite_vf_packet_rxtx.py
> > > @@ -125,6 +125,9 @@ class TestVfPacketRxtx(TestCase):
> > >      def packet_rx_tx(self, driver='default'):
> > >
> > >          if driver == 'igb_uio':
> > > +            # ensure igb_uio driver is installed
> > > +            self.dut.send_expect('(lsmod | grep uio) || (modprobe
> > > + uio)',
> > > '#')
> > > +            self.dut.send_expect('(lsmod | grep igb_uio) ||
> > > (insmod ./%s/kmod/igb_uio.ko)' % self.target,'#')
> > >              self.setup_2pf_2vf_1vm_env(driver='igb_uio')
> > >          else:
> > >              self.setup_2pf_2vf_1vm_env(driver='')
> > > @@ -279,6 +282,7 @@ class TestVfPacketRxtx(TestCase):
> > >          rx_port = tx_port
> > >
> > >          dst_mac = pmd0_vf0_mac
> > > +        self.vm0_testpmd.execute_cmd('clear port stats all')
> > >          self.tester.sendpkt_bg(tx_port, dst_mac)
> > >
> > >          #vf port stop/start can trigger reset action
> > > --
> > > 1.8.3.1

Re: [dts] [PATCH v1] tests/vf_packet_rxtx: fix test case failures in vf_reset

[Herbert Guan]

Hi Marvin,

This seems to the right approach.  Let's abandon this patch, and a new patch dedicated for
"Clear port stats before send/recv test packets" will be  submitted later.

Best regards,
Herbert

> -----Original Message-----
> From: Liu, Yong [mailto:yong.liu@intel.com]
> Sent: Friday, March 2, 2018 16:01
> To: Herbert Guan <Herbert.Guan@arm.com>; dts@dpdk.org
> Cc: nd <nd@arm.com>
> Subject: RE: [dts] [PATCH v1] tests/vf_packet_rxtx: fix test case failures in
> vf_reset
> 
> Herbert,
> We have also incurred your problem in some VF related cases. Look like most
> VF related cases need one default module (igb_uio or vfio noiommu) in VM.
> As default module should be bound with VM, I'd like to add one global option
> for this default module.
> Meanwhile test suite can change the default value in their own configuration.
> 
> Thanks,
> Marvin
> 
> > -----Original Message-----
> > From: Herbert Guan [mailto:Herbert.Guan@arm.com]
> > Sent: Friday, March 02, 2018 2:58 PM
> > To: Liu, Yong <yong.liu@intel.com>; dts@dpdk.org
> > Cc: nd <nd@arm.com>
> > Subject: RE: [dts] [PATCH v1] tests/vf_packet_rxtx: fix test case
> > failures in vf_reset
> >
> > Marvin,
> >
> > When the execution config file selects driver mode to a certain mode
> > (vfio, igb_uio, etc.), then both host and VM NICs will use this mode
> > in case setup.  If some case need VM to use the igb_uio mode,
> > regardless of the driver mode in execution configure file, we need to
> > ensure the required drivers have been installed.
> >
> > If we want to restore the environment, we need to add some extra logic
> > to the case script.  If the uio or igb_uio driver is not installed,
> > then install it and make flag.  This script need to remove installed
> > drivers that were not installed before running the test.
> >
> > Best regards,
> > Herbert
> >
> > > -----Original Message-----
> > > From: Liu, Yong [mailto:yong.liu@intel.com]
> > > Sent: Friday, March 2, 2018 10:08
> > > To: Herbert Guan <Herbert.Guan@arm.com>; dts@dpdk.org
> > > Subject: RE: [dts] [PATCH v1] tests/vf_packet_rxtx: fix test case
> > failures in
> > > vf_reset
> > >
> > > Hi Herbert,
> > > Why required driver is not insmod be previous setup?
> > > As one principle, we want to limit the influence of environment
> > > change
> > in the
> > > suite.
> > >
> > > Thanks,
> > > Marvin
> > >
> > > > -----Original Message-----
> > > > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Herbert Guan
> > > > Sent: Wednesday, February 14, 2018 2:29 PM
> > > > To: dts@dpdk.org
> > > > Cc: Herbert Guan <herbert.guan@arm.com>
> > > > Subject: [dts] [PATCH v1] tests/vf_packet_rxtx: fix test case
> > > > failures in vf_reset
> > > >
> > > > 1) Clear port stats before send/recv test packets
> > > > 2) Ensure igb_uio driver is installed if driver='igb_uio'
> > > >
> > > >
> > > > Signed-off-by: Herbert Guan <herbert.guan@arm.com>
> > > > ---
> > > >  tests/TestSuite_vf_packet_rxtx.py | 4 ++++
> > > >  1 file changed, 4 insertions(+)
> > > >
> > > > diff --git a/tests/TestSuite_vf_packet_rxtx.py
> > > > b/tests/TestSuite_vf_packet_rxtx.py
> > > > index 63f4820..f0b7721 100644
> > > > --- a/tests/TestSuite_vf_packet_rxtx.py
> > > > +++ b/tests/TestSuite_vf_packet_rxtx.py
> > > > @@ -125,6 +125,9 @@ class TestVfPacketRxtx(TestCase):
> > > >      def packet_rx_tx(self, driver='default'):
> > > >
> > > >          if driver == 'igb_uio':
> > > > +            # ensure igb_uio driver is installed
> > > > +            self.dut.send_expect('(lsmod | grep uio) || (modprobe
> > > > + uio)',
> > > > '#')
> > > > +            self.dut.send_expect('(lsmod | grep igb_uio) ||
> > > > (insmod ./%s/kmod/igb_uio.ko)' % self.target,'#')
> > > >              self.setup_2pf_2vf_1vm_env(driver='igb_uio')
> > > >          else:
> > > >              self.setup_2pf_2vf_1vm_env(driver='')
> > > > @@ -279,6 +282,7 @@ class TestVfPacketRxtx(TestCase):
> > > >          rx_port = tx_port
> > > >
> > > >          dst_mac = pmd0_vf0_mac
> > > > +        self.vm0_testpmd.execute_cmd('clear port stats all')
> > > >          self.tester.sendpkt_bg(tx_port, dst_mac)
> > > >
> > > >          #vf port stop/start can trigger reset action
> > > > --
> > > > 1.8.3.1