DPDK CI discussions
 help / color / mirror / Atom feed
From: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
To: Adam Hassick <ahassick@iol.unh.edu>
Cc: Konstantin Ushakov <Konstantin.Ushakov@oktetlabs.ru>,
	Patrick Robb <probb@iol.unh.edu>,
	ci@dpdk.org
Subject: Re: Setting up DPDK PMD Test Suite
Date: Fri, 27 Oct 2023 11:01:56 +0300	[thread overview]
Message-ID: <f9864f9f-d9d0-4992-afd6-a07d0047daad@oktetlabs.ru> (raw)
In-Reply-To: <CAC-YWqiQJUJMc6GjHbkEz8=+Mvnk5XEq=v3g8Kh-iVHxgHkwCA@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 275194 bytes --]

Hi Adam,

fresh ts-rigs-sample has all required bindings to publish logs to 
ts-factory.io. See the top-most commit.
So, you just need tune SSH to use correct user, port and key for 
ts-factory.io. You  can add below settings
to corresponding user .ssh/config or add corresponding options to sftp 
command in scripts/publish_logs/prj/ts-factory/publish:

Host ts-factory.io
         User unh-iol
         Port 56777
         IdentitiesOnly yes
         IdentityFile <path-to-private-key>

The publishing is asynchronous in ts-factory case. i.e. user puts logs 
and cron job checks incoming directory every 10 minutes and publish 
found logs.

You can request publishing when you start testing using --publish option 
or publish previous run testing results using ./scripts/publish_logs 
from dpdk-ethdev-ts sources. PWD should be the directory you run tests from.

Please, let me know how it goes.

Regards,
Andrew.

On 10/26/23 20:44, Adam Hassick wrote:
> Hi Andrew,
>
> I've attached the public key we'd like to use.
>
> Thanks,
> Adam
>
> On Thu, Oct 26, 2023 at 8:19 AM Andrew Rybchenko 
> <andrew.rybchenko@oktetlabs.ru> wrote:
>
>     Hi Adam,
>
>     On 10/25/23 23:27, Adam Hassick wrote:
>>     Hi Andrew,
>>
>>     Sorry about the two week radio silence, we're still trying to
>>     sort out logistics for deployment on our end.
>>
>>     > I've created setup on ts-factory.io <http://ts-factory.io>
>>     which allows to publish logs. Let me know if you'd like to try it
>>     and I'll provide credentials, script and short instruction.
>>
>>     We're interested in publishing some test logs to the ts-factory
>>     Bublik instance in the meantime.
>
>     Please, send me SSH public key which you'd like to use to upload logs.
>     I'll provide helper script and instructions to do it.
>
>     Andrew.
>>
>>     Thanks,
>>     Adam
>>
>>     On Mon, Oct 23, 2023 at 7:11 AM Andrew Rybchenko
>>     <andrew.rybchenko@oktetlabs.ru> wrote:
>>
>>         Hi Adam,
>>
>>         > Now that our test results are in line with yours, we can
>>         begin looking into setting up the production environment.
>>
>>         Please, let me know if you need any help with it or waiting
>>         for an input from me.
>>
>>         Regards,
>>         Andrew.
>>
>>         On 10/10/23 17:09, Adam Hassick wrote:
>>>         Hi Andrew,
>>>
>>>         Thank you for taking a look at our log. Netplan was
>>>         attempting to run DHCP on our test links, and additionally I
>>>         discovered that the NIC firmware was transmitting LLDP
>>>         packets, causing tests to fail in the same way. Now that
>>>         these problems have been fixed, our pass rate on the XL710
>>>         is approximately 91%. Now that our test results are in line
>>>         with yours, we can begin looking into setting up the
>>>         production environment.
>>>
>>>         First, is it possible to run the test agent on ARM hosts?
>>>         Our ARM testbeds have the best topology for running this
>>>         test suite, with separate tester and DUT servers.
>>>
>>>         We are testing this test suite on two x86 development
>>>         servers using the test suite's recommended server topology.
>>>         In contrast, our existing x86 production testbeds which run
>>>         DTS have a single server topology. This single server has
>>>         both the tester NIC and the device under test NIC installed,
>>>         with NUMA node separation between TRex and DPDK. We're going
>>>         to test running the two test agent processes on the
>>>         single-server testbeds if we cannot run this on ARM. Is
>>>         there any reason you can think of that would prevent this
>>>         setup from working?
>>>
>>>         Once we figure out where this can live in production, then
>>>         we will begin setting up log storage, Jenkins integration,
>>>         and Bublik.
>>>
>>>         Thanks,
>>>         Adam
>>>
>>>         On Thu, Oct 5, 2023 at 6:25 AM Andrew Rybchenko
>>>         <andrew.rybchenko@oktetlabs.ru> wrote:
>>>
>>>             Hi Adam,
>>>
>>>             > Do these default to vfio-pci?
>>>
>>>             Yes, vfio-pci is the default.
>>>             However, it does not work in the case of Mellanox which
>>>             uses bifurcated driver. It should mlx5_core for Mellanox
>>>             NICs.
>>>
>>>             > Here is the text log from a run on our Intel XL710
>>>             NICs, with the expected result profile set to the X710.
>>>
>>>             It is hard to analyze all tests using text logs, but I
>>>             definitely see one common problem. Tests receive
>>>             unexpected packets and fail because of it.
>>>             Tests are written very strict from this point of view
>>>             and it brought fruits in the past when HW had bugs.
>>>             Are DUT and tester connected back-to-back on tested
>>>             interfaces or via switch?
>>>             If via switch, is it possible to isolate it from
>>>             everything else?
>>>             If back-to-back, it could be some embedded SW/FW which
>>>             regenerates these packet.
>>>             I definitely see unexpected DHCP packets.
>>>
>>>             > We haven't set up the Jenkins integration yet, however
>>>             if this is required to import the logs then we will
>>>             prioritize that.
>>>
>>>             Unfortunately manual runs do not generate all artifacts
>>>             required to import logs. However, we have almost solved
>>>             it right now. Hopefully we'll finalize it in a day or
>>>             two. I'll let you know when these changes are available.
>>>
>>>             Regards,
>>>             Andrew.
>>>
>>>             On 10/4/23 16:48, Adam Hassick wrote:
>>>>             Hi Andrew,
>>>>
>>>>             Ok, that makes sense. I don't see
>>>>             TE_ENV_H1/H2_DPDK_DRIVER set anywhere in the default
>>>>             configurations for the Intel X710. Do these default to
>>>>             vfio-pci?
>>>>             We have IOMMU enabled on our development testbed, and
>>>>             should be able to bind vfio-pci.
>>>>             Here is the text log from a run on our Intel XL710
>>>>             NICs, with the expected result profile set to the X710.
>>>>             We haven't set up the Jenkins integration yet, however
>>>>             if this is required to import the logs then we will
>>>>             prioritize that.
>>>>             log.txt.tar.gz
>>>>             <https://drive.google.com/file/d/10N5JfxFMP7lNXDBgJeN_z-NL2JNUY7nJ/view?usp=drive_web>
>>>>
>>>>             Thanks,
>>>>             Adam
>>>>
>>>>             On Mon, Sep 18, 2023 at 11:04 AM Andrew Rybchenko
>>>>             <andrew.rybchenko@oktetlabs.ru> wrote:
>>>>
>>>>                 On 9/18/23 17:44, Adam Hassick wrote:
>>>>>                 Hi Andrew and Konstantin,
>>>>>
>>>>>                 Thank you for adding the tester-dial feature, this
>>>>>                 opens up the possibility for us to do CI
>>>>>                 integrated testing in the future.
>>>>>
>>>>>                 Our Mellanox pass rate is similar to yours (about
>>>>>                 ~2400 passing, ~4400 failing), however our Intel
>>>>>                 pass rates are far worse.
>>>>>                 I will try running tests on the XL710 with the
>>>>>                 trc-tags argument set and see if it improves the
>>>>>                 pass rate.
>>>>>                 Another thing I noticed in the results you
>>>>>                 uploaded is that the results are tagged with
>>>>>                 vfio-pci and not i40e.
>>>>>                 Though in the environment dump, the driver on the
>>>>>                 test machine and the DUT are set to use the i40e
>>>>>                 driver. Is this important at all?
>>>>
>>>>                 I think it is a misunderstanding here. There are
>>>>                 two kinds of driver in configuration: net driver
>>>>                 and so-called DPDK driver.
>>>>                 Net driver is a Linux kernel network device driver
>>>>                 used on Tester side.
>>>>                 DPDK driver is a Linux kernel driver to bind device
>>>>                 to to use it with DPDK. So, it is NOT a driver
>>>>                 inside DPDK (drivers/net/*).
>>>>                 In the case of bifurcated driver (like mlx5_core)
>>>>                 it is the same in both cases.
>>>>                 In non-bifurcated case DPDK driver is some UIO
>>>>                 driver(vfio-pci, uio-pci-generic or igb_uio).
>>>>                 Some expectations depend on used UIO. For example,
>>>>                 uio-pci-generic do not support many interrupts
>>>>                 (used by usecases/rx_intr test cases).
>>>>                 That's why we care corresponding TRC tag.
>>>>
>>>>                 TE_ENV_*_DPDK_DRIVER variables should be vfio-pc 
>>>>                 in 710's Intel case. Or uio-pci-generic if IOMMU is
>>>>                 turned off on corresponding machines and Linux
>>>>                 distro does not support VFIO no IOMMU mode.
>>>>
>>>>                 Andrew.
>>>>
>>>>>                 There isn't anything preventing us from pushing
>>>>>                 our results up to the existing Bublik instance
>>>>>                 running at ts-factory.io <http://ts-factory.io>
>>>>>                 that I can think of at the moment.
>>>>>                 We will have to work out how to submit our results
>>>>>                 to your Bublik instance in a controlled and secure
>>>>>                 manner in that case.
>>>>>                 As far as I know we won't need access controls for
>>>>>                 the results themselves. I'll discuss this with
>>>>>                 Patrick and will let you know once we confirm that
>>>>>                 it's fine.
>>>>>
>>>>>                 Thanks,
>>>>>                 Adam
>>>>>
>>>>>                 On Mon, Sep 18, 2023 at 2:26 AM Andrew Rybchenko
>>>>>                 <andrew.rybchenko@oktetlabs.ru> wrote:
>>>>>
>>>>>                     On 9/18/23 09:23, Konstantin Ushakov wrote:
>>>>>>
>>>>>>                     Hi Andrew,
>>>>>>
>>>>>>                     should we always auto-assign the tags or you
>>>>>>                     don’t do it since it slows down (by some
>>>>>>                     seconds) the TE startup?
>>>>>>
>>>>>
>>>>>                     Tags are auto-assigned, but I guess it differs
>>>>>                     in Adam's case since NIC is a bit different.
>>>>>                     Below test will help to understand if it is
>>>>>                     the root cause of very different expectations.
>>>>>                     If pass rate will be close to mine, I'll
>>>>>                     simply update TRC database to share
>>>>>                     expectations for mine NIC and NIC used by Adam.
>>>>>
>>>>>>                     Hi Adam,
>>>>>>
>>>>>>                     I think I second the question from Andrew -
>>>>>>                     happy to help you with the triage so that we
>>>>>>                     get to the same baseline. Do you have a good
>>>>>>                     way for us to share the logs? I.e. say upload
>>>>>>                     to ts-factory if we add strict permissions
>>>>>>                     system so it’s not publishing or any other way.
>>>>>>
>>>>>>                     Thanks,
>>>>>>                     Konstantin
>>>>>>
>>>>>>
>>>>>>                     On 18 Sep 2023, at 9:15, Andrew Rybchenko wrote:
>>>>>>
>>>>>>                         Hi Adam,
>>>>>>
>>>>>>                         I've uploaded fresh testing results to
>>>>>>                         ts-factory.io <http://ts-factory.io> [1]
>>>>>>                         to be on the same page.
>>>>>>
>>>>>>                         I think I know why your and mine results
>>>>>>                         on Intel 710 series NICs differ so much.
>>>>>>                         Testing results expectations database
>>>>>>                         (dpdk-ethdev-ts/trc/*) is filled in in
>>>>>>                         terms of TRC tags.  I.e. expectations
>>>>>>                         depends on TRC tags discovered by helper
>>>>>>                         scripts when testing is started. These
>>>>>>                         tags identify various aspects of what is
>>>>>>                         tested. Ideally expectations should be
>>>>>>                         written in terms of root cause of the
>>>>>>                         expected behaviour. If it is a driver
>>>>>>                         expectations, driver tag should be used.
>>>>>>                         If it is HW limitation, tags with PCI IDs
>>>>>>                         should be used. However, it is not always
>>>>>>                         easy to classify it correctly if you're
>>>>>>                         not involved in driver development. So,
>>>>>>                         in order case expectations for 710's
>>>>>>                         Intel are filled in in terms of PCI IDs.
>>>>>>                         I guess PCI ID differ in your case and
>>>>>>                         that's why expectations filled in for my
>>>>>>                         NIC do not apply to your runs.
>>>>>>
>>>>>>                         Just try to add the following option when
>>>>>>                         you run on your 710's Intel in order to
>>>>>>                         mimic mine and see if it helps to achieve
>>>>>>                         better pass rate.
>>>>>>                         --trc-tag=pci-8086-1572
>>>>>>
>>>>>>                         BTW, fresh TE tag v1.21.0 has improved
>>>>>>                         algorithm to choose tests for
>>>>>>                         --tester-dial option. It should have
>>>>>>                         better coverage now.
>>>>>>
>>>>>>                         Andrew.
>>>>>>
>>>>>>                         [1]
>>>>>>                         https://ts-factory.io/bublik/v2/runs?startDate=2023-09-16&finishDate=2023-09-16&runData=&runDataExpr=&page=1
>>>>>>                         <https://ts-factory.io/bublik/v2/runs?startDate=2023-09-16&finishDate=2023-09-16&runData=&runDataExpr=&page=1>
>>>>>>
>>>>>>                         On 9/13/23 18:45, Andrew Rybchenko wrote:
>>>>>>>                         Hi Adam,
>>>>>>>
>>>>>>>                         I've pushed new TE tag v1.20.0 which
>>>>>>>                         supported a new command-line option
>>>>>>>                         --tester-dial=NUM where NUM is from 0 to
>>>>>>>                         100. it allows to choose percentage of
>>>>>>>                         tests to run. If you want stable set,
>>>>>>>                         you should pass --tester-random-seed=0
>>>>>>>                         (or other integer). It is the first
>>>>>>>                         sketch and we have plans to improve it,
>>>>>>>                         but feedback would be welcome.
>>>>>>>
>>>>>>>                         > Is it needed on the tester?
>>>>>>>
>>>>>>>                         It is hard to say if it is strictly
>>>>>>>                         required for simple tests. However, it
>>>>>>>                         is better to update Tester as well,
>>>>>>>                         since performance tests run DPDK on
>>>>>>>                         Tester as well.
>>>>>>>
>>>>>>>                         > Are there any other manual setup steps
>>>>>>>                         for these devices that I might be missing?
>>>>>>>
>>>>>>>                         I don't remember anything else.
>>>>>>>
>>>>>>>                         I think it is better to get down to
>>>>>>>                         details and take a look at logs. I'm
>>>>>>>                         ready to help with it and explain what's
>>>>>>>                         happening there. May be it will help to
>>>>>>>                         understand if it is a problem with
>>>>>>>                         setup/configuration.
>>>>>>>
>>>>>>>                         Text logs are not very convenient.
>>>>>>>                         Ideally logs should be imported to
>>>>>>>                         bublik, however, manual runs do not
>>>>>>>                         provide all required artifacts right now
>>>>>>>                         (Jenkins jobs generate all required
>>>>>>>                         artifacts).
>>>>>>>                         Other option is 'tmp_raw_log' file
>>>>>>>                         (should be packed to make it smaller)
>>>>>>>                         which could be converted to various log
>>>>>>>                         formats.
>>>>>>>                         Would it be OK for you if I import your
>>>>>>>                         logs to bublik at ts-factory.io
>>>>>>>                         <http://ts-factory.io>? Or is it a
>>>>>>>                         problem that it is publicly available?
>>>>>>>                         Would it help if we add authentication
>>>>>>>                         and access control there?
>>>>>>>
>>>>>>>                         Andrew.
>>>>>>>
>>>>>>>                         On 9/8/23 17:57, Adam Hassick wrote:
>>>>>>>>                         Hi Andrew,
>>>>>>>>
>>>>>>>>                         I have a couple questions about needed
>>>>>>>>                         setup of the NICs for the ethdev test
>>>>>>>>                         suite.
>>>>>>>>
>>>>>>>>                         Our MCX5s and XL710s are failing the
>>>>>>>>                         checkup tests. The pass rate appears to
>>>>>>>>                         be much worse on the XL710s (40 of 73
>>>>>>>>                         tests failed, 3 passed unexpectedly).
>>>>>>>>
>>>>>>>>                         For the XL710s, I've updated the driver
>>>>>>>>                         and NVM versions to match the minimum
>>>>>>>>                         supported versions in the compatibility
>>>>>>>>                         matrix found on the DPDK documentation.
>>>>>>>>                         This did not change the failure rate much.
>>>>>>>>                         For the MCX5s, I've installed the
>>>>>>>>                         latest LTS version of the OFED
>>>>>>>>                         bifurcated driver on the DUT. Is it
>>>>>>>>                         needed on the tester?
>>>>>>>>
>>>>>>>>                         Are there any other manual setup steps
>>>>>>>>                         for these devices that I might be missing?
>>>>>>>>
>>>>>>>>                         Thanks,
>>>>>>>>                         Adam
>>>>>>>>
>>>>>>>>                         On Wed, Sep 6, 2023 at 11:00 AM Adam
>>>>>>>>                         Hassick <ahassick@iol.unh.edu> wrote:
>>>>>>>>
>>>>>>>>                             Hi Andrew,
>>>>>>>>
>>>>>>>>                             Yes, I copied the X710 configs to
>>>>>>>>                             set up XL710 configs. I changed the
>>>>>>>>                             environment variable names from the
>>>>>>>>                             X710 suffix to XL710 suffix in the
>>>>>>>>                             script, and forgot to change them
>>>>>>>>                             in the corresponding environment file.
>>>>>>>>                             That fixed the issue.
>>>>>>>>
>>>>>>>>                             I got the checkup tests working on
>>>>>>>>                             the XL710 now. Most of them are
>>>>>>>>                             failing, which leads me to believe
>>>>>>>>                             this is an issue with our testbed.
>>>>>>>>                             Based on the DPDK documentation for
>>>>>>>>                             i40e, the firmware and driver
>>>>>>>>                             versions are much older than what
>>>>>>>>                             DPDK 22.11 LTS and main prefer, so
>>>>>>>>                             I'll try updating those.
>>>>>>>>
>>>>>>>>                             For now I'm working on getting the
>>>>>>>>                             XL710 checkup tests passing, and
>>>>>>>>                             will pick up getting the E810
>>>>>>>>                             configured properly next. I'll let
>>>>>>>>                             you know if I run into any more
>>>>>>>>                             issues in relation to the test engine.
>>>>>>>>
>>>>>>>>                             Thanks,
>>>>>>>>                             Adam
>>>>>>>>
>>>>>>>>                             On Wed, Sep 6, 2023 at 7:36 AM
>>>>>>>>                             Andrew Rybchenko
>>>>>>>>                             <andrew.rybchenko@oktetlabs.ru> wrote:
>>>>>>>>
>>>>>>>>                                 Hi Adam,
>>>>>>>>
>>>>>>>>                                 On 9/5/23 18:01, Adam Hassick
>>>>>>>>                                 wrote:
>>>>>>>>>                                 Hi Andrew,
>>>>>>>>>
>>>>>>>>>                                 The compilation warning issue
>>>>>>>>>                                 is now resolved. Again, thank
>>>>>>>>>                                 you guys for fixing this for
>>>>>>>>>                                 us. I can run the tests on the
>>>>>>>>>                                 Mellanox CX5s again, however
>>>>>>>>>                                 I'm running into a couple new
>>>>>>>>>                                 issues with running the
>>>>>>>>>                                 prologues on the Intel cards.
>>>>>>>>>
>>>>>>>>>                                 When running testing on the
>>>>>>>>>                                 Intel XL710s, I see this error
>>>>>>>>>                                 appear in the log:
>>>>>>>>>
>>>>>>>>>                                     ERROR  prologue
>>>>>>>>>                                      Environment LIB  14:16:13.650
>>>>>>>>>                                     Too few networks in
>>>>>>>>>                                     available configuration
>>>>>>>>>                                     (0) in comparison with
>>>>>>>>>                                     required (1)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                                 This seems like a trivial
>>>>>>>>>                                 configuration error, perhaps
>>>>>>>>>                                 this is something I need to
>>>>>>>>>                                 set up in ts-rigs. I briefly
>>>>>>>>>                                 searched through the examples
>>>>>>>>>                                 there and didn't see any
>>>>>>>>>                                 mention of how to set up a
>>>>>>>>>                                 network.
>>>>>>>>>                                 I will attach this log just in
>>>>>>>>>                                 case you need more information.
>>>>>>>>
>>>>>>>>                                 Unfortunately logs are
>>>>>>>>                                 insufficient to understand it.
>>>>>>>>                                 I've pushed new tag to TE
>>>>>>>>                                 v1.19.0 which add log message
>>>>>>>>                                 with TE_* environment variables.
>>>>>>>>                                 Most likely something is wrong
>>>>>>>>                                 with variables which are used
>>>>>>>>                                 as conditions when available
>>>>>>>>                                 networks are defined in
>>>>>>>>                                 ts-conf/cs/inc.net_cfg_pci_fns.yml:
>>>>>>>>                                 TE_PCI_INSTANCE_IUT_TST1
>>>>>>>>                                 TE_PCI_INSTANCE_IUT_TST1a
>>>>>>>>                                 TE_PCI_INSTANCE_TST1a_IUT
>>>>>>>>                                 TE_PCI_INSTANCE_TST1_IUT
>>>>>>>>                                 My guess it that you change
>>>>>>>>                                 naming a bit, but script like
>>>>>>>>                                 ts-rigs-sample/scripts/iut.h1-x710
>>>>>>>>                                 is not included or not updated.
>>>>>>>>
>>>>>>>>>                                 There is a different error
>>>>>>>>>                                 when running on the Intel
>>>>>>>>>                                 E810s. It appears to me like
>>>>>>>>>                                 it starts DPDK, does some
>>>>>>>>>                                 configuration inside DPDK and
>>>>>>>>>                                 on the device, and then fails
>>>>>>>>>                                 to bring the device back up.
>>>>>>>>>                                 Since this error seems very
>>>>>>>>>                                 non-trivial, I will also
>>>>>>>>>                                 attach this log.
>>>>>>>>
>>>>>>>>                                 This one is a bit simpler. Few
>>>>>>>>                                 lines after the first ERROR in
>>>>>>>>                                 log I see the following:
>>>>>>>>                                 WARN  RCF DPDK 13:06:00.144
>>>>>>>>                                 ice_program_hw_rx_queue():
>>>>>>>>                                 currently package doesn't
>>>>>>>>                                 support RXDID (22)
>>>>>>>>                                 ice_rx_queue_start(): fail to
>>>>>>>>                                 program RX queue 0
>>>>>>>>                                 ice_dev_start(): fail to start
>>>>>>>>                                 Rx queue 0
>>>>>>>>                                 Device with port_id=0 already
>>>>>>>>                                 stopped
>>>>>>>>
>>>>>>>>                                 It is stdout/stderr from test
>>>>>>>>                                 agent which runs DPDK. Same
>>>>>>>>                                 logs in plain format are
>>>>>>>>                                 available in ta.DPDK file.
>>>>>>>>                                 I'm not an expert here, but I
>>>>>>>>                                 vaguely remember that E810
>>>>>>>>                                 requires correct firmware and
>>>>>>>>                                 DDP to be loaded.
>>>>>>>>                                 There is some information in
>>>>>>>>                                 dpdk/doc/guides/nics/ice.rst.
>>>>>>>>
>>>>>>>>                                 You can try to add
>>>>>>>>                                 --dev-args=safe-mode-support=1
>>>>>>>>                                 command-line option described
>>>>>>>>                                 there.
>>>>>>>>
>>>>>>>>                                 Hope it helps,
>>>>>>>>                                 Andrew.
>>>>>>>>
>>>>>>>>>
>>>>>>>>>                                 Thanks,
>>>>>>>>>                                 Adam
>>>>>>>>>
>>>>>>>>>                                 On Fri, Sep 1, 2023 at 3:59 AM
>>>>>>>>>                                 Andrew Rybchenko
>>>>>>>>>                                 <andrew.rybchenko@oktetlabs.ru>
>>>>>>>>>                                 wrote:
>>>>>>>>>
>>>>>>>>>                                     Hi Adam,
>>>>>>>>>
>>>>>>>>>                                     On 8/31/23 22:38, Adam
>>>>>>>>>                                     Hassick wrote:
>>>>>>>>>>                                     Hi Andrew,
>>>>>>>>>>
>>>>>>>>>>                                     I have one additional
>>>>>>>>>>                                     question as well: Does
>>>>>>>>>>                                     the test engine support
>>>>>>>>>>                                     running tests on two
>>>>>>>>>>                                     ARMv8 test agents?
>>>>>>>>>>
>>>>>>>>>>                                         1. We'll sort out
>>>>>>>>>>                                         warnings this week.
>>>>>>>>>>                                         Thanks for heads up.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                                     Great. Let me know when
>>>>>>>>>>                                     that's fixed.
>>>>>>>>>
>>>>>>>>>                                     Done. We also fixed a
>>>>>>>>>                                     number of warnings in TE.
>>>>>>>>>                                     Also we fixed root test
>>>>>>>>>                                     package name to be
>>>>>>>>>                                     consistent with the
>>>>>>>>>                                     repository name.
>>>>>>>>>
>>>>>>>>>>                                         Support for old LTS
>>>>>>>>>>                                         branches was dropped
>>>>>>>>>>                                         some time ago, but in
>>>>>>>>>>                                         the future it is
>>>>>>>>>>                                         definitely possible
>>>>>>>>>>                                         to keep it for new
>>>>>>>>>>                                         LTS branches. I think
>>>>>>>>>>                                         22.11 is supported,
>>>>>>>>>>                                         but I'm not sure
>>>>>>>>>>                                         about older LTS releases.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                                     Good to know.
>>>>>>>>>>
>>>>>>>>>>                                         2. You can add
>>>>>>>>>>                                         command-line option
>>>>>>>>>>                                         --sanity to run tests
>>>>>>>>>>                                         marked with
>>>>>>>>>>                                         TEST_HARNESS_SANITY
>>>>>>>>>>                                         requirement (see
>>>>>>>>>>                                         dpdk-ethdev-ts/scripts/run.sh
>>>>>>>>>>                                         and grep
>>>>>>>>>>                                         TEST_HARNESS_SANITY
>>>>>>>>>>                                         dpdk-ethdev-ts to see
>>>>>>>>>>                                         which tests are
>>>>>>>>>>                                         marked). Yes, there
>>>>>>>>>>                                         is a space for
>>>>>>>>>>                                         terminology
>>>>>>>>>>                                         improvement here.
>>>>>>>>>>                                         We'll do it.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                                     Done. Now it is called
>>>>>>>>>                                     --checkup.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                                         Also it takes a lot
>>>>>>>>>>                                         of time because of
>>>>>>>>>>                                         failures and tests
>>>>>>>>>>                                         which wait for some
>>>>>>>>>>                                         timeout.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                                     That makes sense to me.
>>>>>>>>>>                                     We'll use the time to
>>>>>>>>>>                                     complete tests on virtio
>>>>>>>>>>                                     or the Intel devices as a
>>>>>>>>>>                                     reference for how long
>>>>>>>>>>                                     the tests really take to
>>>>>>>>>>                                     complete.
>>>>>>>>>>                                     We will explore the
>>>>>>>>>>                                     possibility of
>>>>>>>>>>                                     periodically running the
>>>>>>>>>>                                     sanity tests for patches.
>>>>>>>>>
>>>>>>>>>                                     I'll double-check and let
>>>>>>>>>                                     you know how long entire
>>>>>>>>>                                     TS runs on Intel X710,
>>>>>>>>>                                     E810, Mellanox CX5 and
>>>>>>>>>                                     virtio net. Just to ensure
>>>>>>>>>                                     that time observed in your
>>>>>>>>>                                     case looks the same.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                                         The test harness can
>>>>>>>>>>                                         provide coverage
>>>>>>>>>>                                         reports based on
>>>>>>>>>>                                         gcov, but I'm not
>>>>>>>>>>                                         sure what you mean by
>>>>>>>>>>                                         a "dial" to control
>>>>>>>>>>                                         test coverage.
>>>>>>>>>>                                         Provided reports are
>>>>>>>>>>                                         rather for human to
>>>>>>>>>>                                         analyze.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                                     The general idea is to
>>>>>>>>>>                                     have some kind of
>>>>>>>>>>                                     parameter on the test
>>>>>>>>>>                                     suite, which could be an
>>>>>>>>>>                                     integer ranging from zero
>>>>>>>>>>                                     to ten, that controls how
>>>>>>>>>>                                     many tests are run based
>>>>>>>>>>                                     on how important the test is.
>>>>>>>>>>
>>>>>>>>>>                                     Similar to how some
>>>>>>>>>>                                     command line interfaces
>>>>>>>>>>                                     provide a verbosity level
>>>>>>>>>>                                     parameter (some number of
>>>>>>>>>>                                     "-v" arguments) to
>>>>>>>>>>                                     control the importance of
>>>>>>>>>>                                     the information in the log.
>>>>>>>>>>                                     The verbosity level zero
>>>>>>>>>>                                     only prints very
>>>>>>>>>>                                     important log messages,
>>>>>>>>>>                                     while ten prints everything.
>>>>>>>>>>
>>>>>>>>>>                                     In much the same manner
>>>>>>>>>>                                     as above, this "dial"
>>>>>>>>>>                                     parameter controls what
>>>>>>>>>>                                     tests are run and with
>>>>>>>>>>                                     what parameters based on
>>>>>>>>>>                                     how important those tests
>>>>>>>>>>                                     and test parameter
>>>>>>>>>>                                     combinations are.
>>>>>>>>>>                                     Coverage Level zero tells
>>>>>>>>>>                                     the suite to run a very
>>>>>>>>>>                                     basic set of important
>>>>>>>>>>                                     tests, with minimal
>>>>>>>>>>                                     parameterization. This
>>>>>>>>>>                                     mode would take only
>>>>>>>>>>                                     ~5-10 minutes to run.
>>>>>>>>>>                                     In contrast, Coverage
>>>>>>>>>>                                     Level ten includes all
>>>>>>>>>>                                     the edge cases, every
>>>>>>>>>>                                     combination of test
>>>>>>>>>>                                     parameters, everything
>>>>>>>>>>                                     the test suite can do,
>>>>>>>>>>                                     which takes the normal
>>>>>>>>>>                                     several hours to run.
>>>>>>>>>>                                     The values 1 - 9 are
>>>>>>>>>>                                     between those two
>>>>>>>>>>                                     extremes, allowing the
>>>>>>>>>>                                     user to get a gradient of
>>>>>>>>>>                                     test coverage in the
>>>>>>>>>>                                     results and to limit the
>>>>>>>>>>                                     running time.
>>>>>>>>>>
>>>>>>>>>>                                     Then we could, for
>>>>>>>>>>                                     example, run the "run.sh"
>>>>>>>>>>                                     with a level of 2 or 3
>>>>>>>>>>                                     for incoming patches that
>>>>>>>>>>                                     need quick results, and
>>>>>>>>>>                                     with a level of 10 for
>>>>>>>>>>                                     the less often run
>>>>>>>>>>                                     periodic tests performed
>>>>>>>>>>                                     on main or LTS branches.
>>>>>>>>>
>>>>>>>>>                                     Understood now. Thanks a
>>>>>>>>>                                     lot for the idea. We'll
>>>>>>>>>                                     discuss it and come back.
>>>>>>>>>
>>>>>>>>>>                                         3. Yes, really many
>>>>>>>>>>                                         tests on Mellanox CX5
>>>>>>>>>>                                         NICs report
>>>>>>>>>>                                         unexpected testing
>>>>>>>>>>                                         results.
>>>>>>>>>>                                         Unfortunately it is
>>>>>>>>>>                                         time consuming to
>>>>>>>>>>                                         fill in expectations
>>>>>>>>>>                                         database since it is
>>>>>>>>>>                                         necessary to analyze
>>>>>>>>>>                                         testing results and
>>>>>>>>>>                                         classify if it is a
>>>>>>>>>>                                         bug or just
>>>>>>>>>>                                         acceptable behaviour
>>>>>>>>>>                                         aspect.
>>>>>>>>>>
>>>>>>>>>>                                         Bublik allows to
>>>>>>>>>>                                         compare results of
>>>>>>>>>>                                         two runs. It is
>>>>>>>>>>                                         useful for human, but
>>>>>>>>>>                                         still not good for
>>>>>>>>>>                                         automation.
>>>>>>>>>>
>>>>>>>>>>                                         I have local patch
>>>>>>>>>>                                         for mlx5 driver which
>>>>>>>>>>                                         reports Tx ring size
>>>>>>>>>>                                         maximum. It makes
>>>>>>>>>>                                         pass rate higher. It
>>>>>>>>>>                                         is a problem for test
>>>>>>>>>>                                         harness that mlx5
>>>>>>>>>>                                         does not report
>>>>>>>>>>                                         limits right now.
>>>>>>>>>>
>>>>>>>>>>                                         Pass rate on Intel
>>>>>>>>>>                                         X710 is about 92% on
>>>>>>>>>>                                         my test rig. Pass
>>>>>>>>>>                                         rate on virtio net is
>>>>>>>>>>                                         99% right now and
>>>>>>>>>>                                         could be done 100%
>>>>>>>>>>                                         easily (just one
>>>>>>>>>>                                         thing to fix in
>>>>>>>>>>                                         expectations).
>>>>>>>>>>
>>>>>>>>>>                                         I think logs storage
>>>>>>>>>>                                         setup is essential
>>>>>>>>>>                                         for logs analysis. Of
>>>>>>>>>>                                         course, you can
>>>>>>>>>>                                         request HTML logs
>>>>>>>>>>                                         when you run tests
>>>>>>>>>>                                         (--log-html=html) or
>>>>>>>>>>                                         generate after run
>>>>>>>>>>                                         using
>>>>>>>>>>                                         dpdk-ethdev-ts/scripts/html-log.sh
>>>>>>>>>>                                         and open index.html
>>>>>>>>>>                                         in a browser, but
>>>>>>>>>>                                         logs storage makes it
>>>>>>>>>>                                         more convenient.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                                     We are interested in
>>>>>>>>>>                                     setting up Bublik,
>>>>>>>>>>                                     potentially as an
>>>>>>>>>>                                     externally-facing
>>>>>>>>>>                                     component, once we have
>>>>>>>>>>                                     our process of running
>>>>>>>>>>                                     the test suite stabilized.
>>>>>>>>>>                                     Once we are able to run
>>>>>>>>>>                                     the test suite again,
>>>>>>>>>>                                     I'll see what the pass
>>>>>>>>>>                                     rate is on our other
>>>>>>>>>>                                     hardware.
>>>>>>>>>>                                     Good to know that it
>>>>>>>>>>                                     isn't an issue with our
>>>>>>>>>>                                     dev testbed causing the
>>>>>>>>>>                                     high fail rate.
>>>>>>>>>>
>>>>>>>>>>                                     For Intel hardware, we
>>>>>>>>>>                                     have an XL710 and an
>>>>>>>>>>                                     Intel E810-C in our
>>>>>>>>>>                                     development testbed.
>>>>>>>>>>                                     Although they are
>>>>>>>>>>                                     slightly different
>>>>>>>>>>                                     devices, ideally the pass
>>>>>>>>>>                                     rate will be identical or
>>>>>>>>>>                                     similar. I have yet to
>>>>>>>>>>                                     set up a VM pair for
>>>>>>>>>>                                     virtio, but we will soon.
>>>>>>>>>>
>>>>>>>>>>                                         Latest version of
>>>>>>>>>>                                         test-environment has
>>>>>>>>>>                                         examples of our CGI
>>>>>>>>>>                                         scripts which we use
>>>>>>>>>>                                         for log storage (see
>>>>>>>>>>                                         tools/log_server/README.md).
>>>>>>>>>>
>>>>>>>>>>                                         Also all bits for
>>>>>>>>>>                                         Jenkins setup are
>>>>>>>>>>                                         available. See
>>>>>>>>>>                                         dpdk-ethdev-ts/jenkins/README.md
>>>>>>>>>>                                         and examples of
>>>>>>>>>>                                         jenkins files in
>>>>>>>>>>                                         ts-rigs-sample.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                                     Jenkins integration,
>>>>>>>>>>                                     setting up production rig
>>>>>>>>>>                                     configurations, and
>>>>>>>>>>                                     permanent log storage
>>>>>>>>>>                                     will be our next steps
>>>>>>>>>>                                     once I am able to run the
>>>>>>>>>>                                     tests again.
>>>>>>>>>>                                     Unless there is an easy
>>>>>>>>>>                                     way to have meson not
>>>>>>>>>>                                     pass "-Werror" into GCC.
>>>>>>>>>>                                     Then I would be able to
>>>>>>>>>>                                     run the test suite.
>>>>>>>>>
>>>>>>>>>                                     Hopefully it is resolved now.
>>>>>>>>>
>>>>>>>>>                                     I thought a bit more about
>>>>>>>>>                                     your usecase for Jenkins.
>>>>>>>>>                                     I'm not 100% sure that
>>>>>>>>>                                     existing pipelines are
>>>>>>>>>                                     convenient for your usecase.
>>>>>>>>>                                     Fill free to ask questions
>>>>>>>>>                                     when you are on it.
>>>>>>>>>
>>>>>>>>>                                     Thanks,
>>>>>>>>>                                     Andrew.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                                     Thanks,
>>>>>>>>>>                                     Adam
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                                         On 8/29/23 17:02,
>>>>>>>>>>                                         Adam Hassick wrote:
>>>>>>>>>>>                                         Hi Andrew,
>>>>>>>>>>>
>>>>>>>>>>>                                         That fix seems to
>>>>>>>>>>>                                         have resolved the
>>>>>>>>>>>                                         issue, thanks for
>>>>>>>>>>>                                         the quick turnaround
>>>>>>>>>>>                                         time on that patch.
>>>>>>>>>>>                                         Now that we have the
>>>>>>>>>>>                                         RCF timeout issue
>>>>>>>>>>>                                         resolved, there are
>>>>>>>>>>>                                         a few other
>>>>>>>>>>>                                         questions and issues
>>>>>>>>>>>                                         that we have about
>>>>>>>>>>>                                         the tests themselves.
>>>>>>>>>>>
>>>>>>>>>>>                                         1. The test suite
>>>>>>>>>>>                                         fails to build with
>>>>>>>>>>>                                         a couple warnings.
>>>>>>>>>>>
>>>>>>>>>>>                                         Below is the stderr
>>>>>>>>>>>                                         log from compilation:
>>>>>>>>>>>
>>>>>>>>>>>                                             FAILED:
>>>>>>>>>>>                                             lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o
>>>>>>>>>>>                                             cc
>>>>>>>>>>>                                             -Ilib/76b5a35@@ts_dpdk_pmd@sta
>>>>>>>>>>>                                             -Ilib
>>>>>>>>>>>                                             -I../../lib
>>>>>>>>>>>                                             -I/opt/tsf/dpdk-ethdev-ts/ts/inst/default/include
>>>>>>>>>>>                                             -fdiagnostics-color=always
>>>>>>>>>>>                                             -pipe
>>>>>>>>>>>                                             -D_FILE_OFFSET_BITS=64
>>>>>>>>>>>                                             -Wall
>>>>>>>>>>>                                             -Winvalid-pch
>>>>>>>>>>>                                             -Werror -g
>>>>>>>>>>>                                             -D_GNU_SOURCE
>>>>>>>>>>>                                             -O0 -ggdb -Wall
>>>>>>>>>>>                                             -W -fPIC -MD -MQ
>>>>>>>>>>>                                             'lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o'
>>>>>>>>>>>                                             -MF
>>>>>>>>>>>                                             'lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o.d'
>>>>>>>>>>>                                             -o
>>>>>>>>>>>                                             'lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o'
>>>>>>>>>>>                                             -c
>>>>>>>>>>>                                             ../../lib/dpdk_pmd_ts.c
>>>>>>>>>>>                                             ../../lib/dpdk_pmd_ts.c:
>>>>>>>>>>>                                             In function
>>>>>>>>>>>                                             ‘test_create_traffic_generator_params’:
>>>>>>>>>>>                                             ../../lib/dpdk_pmd_ts.c:5577:5:
>>>>>>>>>>>                                             error: format
>>>>>>>>>>>                                             not a string
>>>>>>>>>>>                                             literal and no
>>>>>>>>>>>                                             format arguments
>>>>>>>>>>>                                             [-Werror=format-security]
>>>>>>>>>>>                                             5577 |     rc =
>>>>>>>>>>>                                             te_kvpair_add(result,
>>>>>>>>>>>                                             buf, mode);
>>>>>>>>>>>                                             |     ^~
>>>>>>>>>>>                                             cc1: all
>>>>>>>>>>>                                             warnings being
>>>>>>>>>>>                                             treated as errors
>>>>>>>>>>>                                             ninja: build
>>>>>>>>>>>                                             stopped:
>>>>>>>>>>>                                             subcommand failed.
>>>>>>>>>>>                                             ninja: Entering
>>>>>>>>>>>                                             directory `.'
>>>>>>>>>>>                                             FAILED:
>>>>>>>>>>>                                             lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o
>>>>>>>>>>>                                             cc
>>>>>>>>>>>                                             -Ilib/76b5a35@@ts_dpdk_pmd@sta
>>>>>>>>>>>                                             -Ilib
>>>>>>>>>>>                                             -I../../lib
>>>>>>>>>>>                                             -I/opt/tsf/dpdk-ethdev-ts/ts/inst/default/include
>>>>>>>>>>>                                             -fdiagnostics-color=always
>>>>>>>>>>>                                             -pipe
>>>>>>>>>>>                                             -D_FILE_OFFSET_BITS=64
>>>>>>>>>>>                                             -Wall
>>>>>>>>>>>                                             -Winvalid-pch
>>>>>>>>>>>                                             -Werror -g
>>>>>>>>>>>                                             -D_GNU_SOURCE
>>>>>>>>>>>                                             -O0 -ggdb -Wall
>>>>>>>>>>>                                             -W -fPIC -MD -MQ
>>>>>>>>>>>                                             'lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o'
>>>>>>>>>>>                                             -MF
>>>>>>>>>>>                                             'lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o.d'
>>>>>>>>>>>                                             -o
>>>>>>>>>>>                                             'lib/76b5a35@@ts_dpdk_pmd@sta/dpdk_pmd_ts.c.o'
>>>>>>>>>>>                                             -c
>>>>>>>>>>>                                             ../../lib/dpdk_pmd_ts.c
>>>>>>>>>>>                                             ../../lib/dpdk_pmd_ts.c:
>>>>>>>>>>>                                             In function
>>>>>>>>>>>                                             ‘test_create_traffic_generator_params’:
>>>>>>>>>>>                                             ../../lib/dpdk_pmd_ts.c:5577:5:
>>>>>>>>>>>                                             error: format
>>>>>>>>>>>                                             not a string
>>>>>>>>>>>                                             literal and no
>>>>>>>>>>>                                             format arguments
>>>>>>>>>>>                                             [-Werror=format-security]
>>>>>>>>>>>                                             5577 |     rc =
>>>>>>>>>>>                                             te_kvpair_add(result,
>>>>>>>>>>>                                             buf, mode);
>>>>>>>>>>>                                             |     ^~
>>>>>>>>>>>                                             cc1: all
>>>>>>>>>>>                                             warnings being
>>>>>>>>>>>                                             treated as errors
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                                         This error wasn't
>>>>>>>>>>>                                         occurring last week,
>>>>>>>>>>>                                         which was the last
>>>>>>>>>>>                                         time I ran the tests.
>>>>>>>>>>>                                         The TE host and the
>>>>>>>>>>>                                         DUT have GCC v9.4.0
>>>>>>>>>>>                                         installed, and the
>>>>>>>>>>>                                         tester has GCC
>>>>>>>>>>>                                         v11.4.0 installed,
>>>>>>>>>>>                                         if this information
>>>>>>>>>>>                                         is helpful.
>>>>>>>>>>>
>>>>>>>>>>>                                         2. On the Mellanox
>>>>>>>>>>>                                         CX5s, there are over
>>>>>>>>>>>                                         6,000 tests run,
>>>>>>>>>>>                                         which collectively
>>>>>>>>>>>                                         take around 9 hours.
>>>>>>>>>>>                                         Is it possible, and
>>>>>>>>>>>                                         would it make sense,
>>>>>>>>>>>                                         to lower the test
>>>>>>>>>>>                                         coverage and have
>>>>>>>>>>>                                         the test suite run
>>>>>>>>>>>                                         faster?
>>>>>>>>>>>
>>>>>>>>>>>                                         For some context, we
>>>>>>>>>>>                                         run immediate
>>>>>>>>>>>                                         testing on incoming
>>>>>>>>>>>                                         patches for DPDK
>>>>>>>>>>>                                         main and development
>>>>>>>>>>>                                         branches, as well as
>>>>>>>>>>>                                         periodic test runs
>>>>>>>>>>>                                         on the main, stable,
>>>>>>>>>>>                                         and LTS branches.
>>>>>>>>>>>                                         For us to consider
>>>>>>>>>>>                                         including this test
>>>>>>>>>>>                                         suite as part of our
>>>>>>>>>>>                                         immediate testing on
>>>>>>>>>>>                                         patches, we would
>>>>>>>>>>>                                         have to reduce the
>>>>>>>>>>>                                         test coverage to the
>>>>>>>>>>>                                         most important tests.
>>>>>>>>>>>                                         This is primarily to
>>>>>>>>>>>                                         reduce the testing
>>>>>>>>>>>                                         time to, for
>>>>>>>>>>>                                         example, less than
>>>>>>>>>>>                                         30 minutes. Testing
>>>>>>>>>>>                                         on patches can't
>>>>>>>>>>>                                         take too long
>>>>>>>>>>>                                         because the lab can
>>>>>>>>>>>                                         receive numerous
>>>>>>>>>>>                                         patches each day,
>>>>>>>>>>>                                         which each require
>>>>>>>>>>>                                         individual testing runs.
>>>>>>>>>>>
>>>>>>>>>>>                                         At what frequency we
>>>>>>>>>>>                                         run these tests, and
>>>>>>>>>>>                                         on what, still needs
>>>>>>>>>>>                                         to be discussed with
>>>>>>>>>>>                                         the DPDK community,
>>>>>>>>>>>                                         but it would be nice
>>>>>>>>>>>                                         to know if the test
>>>>>>>>>>>                                         suite had a "dial"
>>>>>>>>>>>                                         to control the
>>>>>>>>>>>                                         testing coverage.
>>>>>>>>>>>
>>>>>>>>>>>                                         3. We see a lot of
>>>>>>>>>>>                                         test failures on our
>>>>>>>>>>>                                         Mellanox CX5 NICs.
>>>>>>>>>>>                                         Around 2,300 of
>>>>>>>>>>>                                         ~6,600 tests passed.
>>>>>>>>>>>                                         Is there anything we
>>>>>>>>>>>                                         can do to diagnose
>>>>>>>>>>>                                         these test failures?
>>>>>>>>>>>
>>>>>>>>>>>                                         Thanks,
>>>>>>>>>>>                                         Adam
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                                         On Tue, Aug 29, 2023
>>>>>>>>>>>                                         at 8:07 AM Andrew
>>>>>>>>>>>                                         Rybchenko
>>>>>>>>>>>                                         <andrew.rybchenko@oktetlabs.ru>
>>>>>>>>>>>                                         wrote:
>>>>>>>>>>>
>>>>>>>>>>>                                             Hi Adam,
>>>>>>>>>>>
>>>>>>>>>>>                                             I've pushed the
>>>>>>>>>>>                                             fix in main
>>>>>>>>>>>                                             branch and a new
>>>>>>>>>>>                                             tag v1.18.1. It
>>>>>>>>>>>                                             should solve the
>>>>>>>>>>>                                             problem with
>>>>>>>>>>>                                             IPv6 address
>>>>>>>>>>>                                             from DNS.
>>>>>>>>>>>
>>>>>>>>>>>                                             Andrew.
>>>>>>>>>>>
>>>>>>>>>>>                                             On 8/29/23
>>>>>>>>>>>                                             00:05, Andrew
>>>>>>>>>>>                                             Rybchenko wrote:
>>>>>>>>>>>>                                             Hi Adam,
>>>>>>>>>>>>
>>>>>>>>>>>>                                             > Does the test
>>>>>>>>>>>>                                             engine prefer
>>>>>>>>>>>>                                             to use IPv6
>>>>>>>>>>>>                                             over IPv4 for
>>>>>>>>>>>>                                             initiating the
>>>>>>>>>>>>                                             RCF connection
>>>>>>>>>>>>                                             to the test bed
>>>>>>>>>>>>                                             hosts? And if
>>>>>>>>>>>>                                             so, is there a
>>>>>>>>>>>>                                             way to force it
>>>>>>>>>>>>                                             to use IPv4?
>>>>>>>>>>>>
>>>>>>>>>>>>                                             Brilliant idea.
>>>>>>>>>>>>                                             If DNS returns
>>>>>>>>>>>>                                             both IPv4 and
>>>>>>>>>>>>                                             IPv6 addresses
>>>>>>>>>>>>                                             in your case, I
>>>>>>>>>>>>                                             guess it is the
>>>>>>>>>>>>                                             root cause of
>>>>>>>>>>>>                                             the problem.
>>>>>>>>>>>>                                             Of course, it
>>>>>>>>>>>>                                             is TE problem
>>>>>>>>>>>>                                             since I see
>>>>>>>>>>>>                                             really weird
>>>>>>>>>>>>                                             code in
>>>>>>>>>>>>                                             lib/comm_net_engine/comm_net_engine.c
>>>>>>>>>>>>                                             line 135.
>>>>>>>>>>>>
>>>>>>>>>>>>                                             I've pushed fix
>>>>>>>>>>>>                                             to the branch
>>>>>>>>>>>>                                             user/arybchik/fix_ipv4_only
>>>>>>>>>>>>                                             in
>>>>>>>>>>>>                                             ts-factory/test-environment
>>>>>>>>>>>>                                             repository.
>>>>>>>>>>>>                                             Please, try.
>>>>>>>>>>>>
>>>>>>>>>>>>                                             It is late
>>>>>>>>>>>>                                             night fix with
>>>>>>>>>>>>                                             minimal testing
>>>>>>>>>>>>                                             and no review.
>>>>>>>>>>>>                                             I'll pass it
>>>>>>>>>>>>                                             through review
>>>>>>>>>>>>                                             process
>>>>>>>>>>>>                                             tomorrow and
>>>>>>>>>>>>                                             hopefully it
>>>>>>>>>>>>                                             will be
>>>>>>>>>>>>                                             released in
>>>>>>>>>>>>                                             one-two days.
>>>>>>>>>>>>
>>>>>>>>>>>>                                             Andrew.
>>>>>>>>>>>>
>>>>>>>>>>>>                                             On 8/28/23
>>>>>>>>>>>>                                             18:02, Adam
>>>>>>>>>>>>                                             Hassick wrote:
>>>>>>>>>>>>>                                             Hi Andrew,
>>>>>>>>>>>>>
>>>>>>>>>>>>>                                             We have yet to
>>>>>>>>>>>>>                                             notice a
>>>>>>>>>>>>>                                             distinct
>>>>>>>>>>>>>                                             pattern with
>>>>>>>>>>>>>                                             the failures.
>>>>>>>>>>>>>                                             Sometimes, the
>>>>>>>>>>>>>                                             RCF will start
>>>>>>>>>>>>>                                             and connect
>>>>>>>>>>>>>                                             without issue
>>>>>>>>>>>>>                                             a few times in
>>>>>>>>>>>>>                                             a row before
>>>>>>>>>>>>>                                             failing to
>>>>>>>>>>>>>                                             connect again.
>>>>>>>>>>>>>                                             Once the issue
>>>>>>>>>>>>>                                             begins to
>>>>>>>>>>>>>                                             occur, neither
>>>>>>>>>>>>>                                             rebooting all
>>>>>>>>>>>>>                                             of the hosts
>>>>>>>>>>>>>                                             (test engine
>>>>>>>>>>>>>                                             VM, tester,
>>>>>>>>>>>>>                                             IUT) or
>>>>>>>>>>>>>                                             deleting all
>>>>>>>>>>>>>                                             of the build
>>>>>>>>>>>>>                                             directories
>>>>>>>>>>>>>                                             (suites,
>>>>>>>>>>>>>                                             agents, inst)
>>>>>>>>>>>>>                                             and rebooting
>>>>>>>>>>>>>                                             the hosts
>>>>>>>>>>>>>                                             afterward
>>>>>>>>>>>>>                                             resolves the
>>>>>>>>>>>>>                                             issue. When it
>>>>>>>>>>>>>                                             begins working
>>>>>>>>>>>>>                                             again seems
>>>>>>>>>>>>>                                             very arbitrary
>>>>>>>>>>>>>                                             to us.
>>>>>>>>>>>>>
>>>>>>>>>>>>>                                             I do usually
>>>>>>>>>>>>>                                             try to
>>>>>>>>>>>>>                                             terminate the
>>>>>>>>>>>>>                                             test engine
>>>>>>>>>>>>>                                             with Ctrl+C,
>>>>>>>>>>>>>                                             but when it
>>>>>>>>>>>>>                                             hangs while
>>>>>>>>>>>>>                                             trying to
>>>>>>>>>>>>>                                             start RCF,
>>>>>>>>>>>>>                                             that does not
>>>>>>>>>>>>>                                             work.
>>>>>>>>>>>>>
>>>>>>>>>>>>>                                             Does the test
>>>>>>>>>>>>>                                             engine prefer
>>>>>>>>>>>>>                                             to use IPv6
>>>>>>>>>>>>>                                             over IPv4 for
>>>>>>>>>>>>>                                             initiating the
>>>>>>>>>>>>>                                             RCF connection
>>>>>>>>>>>>>                                             to the test
>>>>>>>>>>>>>                                             bed hosts? And
>>>>>>>>>>>>>                                             if so, is
>>>>>>>>>>>>>                                             there a way to
>>>>>>>>>>>>>                                             force it to
>>>>>>>>>>>>>                                             use IPv4?
>>>>>>>>>>>>>
>>>>>>>>>>>>>                                              - Adam
>>>>>>>>>>>>>
>>>>>>>>>>>>>                                             On Fri, Aug
>>>>>>>>>>>>>                                             25, 2023 at
>>>>>>>>>>>>>                                             1:35 PM Andrew
>>>>>>>>>>>>>                                             Rybchenko
>>>>>>>>>>>>>                                             <andrew.rybchenko@oktetlabs.ru>
>>>>>>>>>>>>>                                             wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>                                                 > I'll
>>>>>>>>>>>>>                                                 double-check
>>>>>>>>>>>>>                                                 test
>>>>>>>>>>>>>                                                 engine on
>>>>>>>>>>>>>                                                 Ubuntu
>>>>>>>>>>>>>                                                 20.04 and
>>>>>>>>>>>>>                                                 Ubuntu 22.04.
>>>>>>>>>>>>>
>>>>>>>>>>>>>                                                 Done. It
>>>>>>>>>>>>>                                                 works fine
>>>>>>>>>>>>>                                                 for me
>>>>>>>>>>>>>                                                 without
>>>>>>>>>>>>>                                                 any issues.
>>>>>>>>>>>>>
>>>>>>>>>>>>>                                                 Have you
>>>>>>>>>>>>>                                                 noticed
>>>>>>>>>>>>>                                                 any
>>>>>>>>>>>>>                                                 pattern
>>>>>>>>>>>>>                                                 when it
>>>>>>>>>>>>>                                                 works or
>>>>>>>>>>>>>                                                 does not work?
>>>>>>>>>>>>>                                                 May be it
>>>>>>>>>>>>>                                                 is a
>>>>>>>>>>>>>                                                 problem of
>>>>>>>>>>>>>                                                 not clean
>>>>>>>>>>>>>                                                 state
>>>>>>>>>>>>>                                                 after
>>>>>>>>>>>>>                                                 termination?
>>>>>>>>>>>>>                                                 Does it
>>>>>>>>>>>>>                                                 work fine
>>>>>>>>>>>>>                                                 the first
>>>>>>>>>>>>>                                                 time after
>>>>>>>>>>>>>                                                 DUTs reboot?
>>>>>>>>>>>>>                                                 How do you
>>>>>>>>>>>>>                                                 terminate
>>>>>>>>>>>>>                                                 testing?
>>>>>>>>>>>>>                                                 It should
>>>>>>>>>>>>>                                                 be done
>>>>>>>>>>>>>                                                 using
>>>>>>>>>>>>>                                                 Ctrl+C in
>>>>>>>>>>>>>                                                 terminal
>>>>>>>>>>>>>                                                 where you
>>>>>>>>>>>>>                                                 execute
>>>>>>>>>>>>>                                                 run.sh
>>>>>>>>>>>>>                                                 command.
>>>>>>>>>>>>>                                                  In this
>>>>>>>>>>>>>                                                 case it
>>>>>>>>>>>>>                                                 should
>>>>>>>>>>>>>                                                 shutdown
>>>>>>>>>>>>>                                                 gracefully
>>>>>>>>>>>>>                                                 and close
>>>>>>>>>>>>>                                                 all test
>>>>>>>>>>>>>                                                 agents and
>>>>>>>>>>>>>                                                 engine
>>>>>>>>>>>>>                                                 applications.
>>>>>>>>>>>>>
>>>>>>>>>>>>>                                                 (I'm
>>>>>>>>>>>>>                                                 trying to
>>>>>>>>>>>>>                                                 understand
>>>>>>>>>>>>>                                                 why you've
>>>>>>>>>>>>>                                                 seen many
>>>>>>>>>>>>>                                                 test agent
>>>>>>>>>>>>>                                                 processes.
>>>>>>>>>>>>>                                                 It should
>>>>>>>>>>>>>                                                 not happen.)
>>>>>>>>>>>>>
>>>>>>>>>>>>>                                                 Andrew.
>>>>>>>>>>>>>
>>>>>>>>>>>>>                                                 On 8/25/23
>>>>>>>>>>>>>                                                 17:41,
>>>>>>>>>>>>>                                                 Andrew
>>>>>>>>>>>>>                                                 Rybchenko
>>>>>>>>>>>>>                                                 wrote:
>>>>>>>>>>>>>>                                                 On
>>>>>>>>>>>>>>                                                 8/25/23
>>>>>>>>>>>>>>                                                 17:06,
>>>>>>>>>>>>>>                                                 Adam
>>>>>>>>>>>>>>                                                 Hassick
>>>>>>>>>>>>>>                                                 wrote:
>>>>>>>>>>>>>>>                                                 Hi Andrew,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>                                                 Two of
>>>>>>>>>>>>>>>                                                 our
>>>>>>>>>>>>>>>                                                 systems
>>>>>>>>>>>>>>>                                                 (the
>>>>>>>>>>>>>>>                                                 Test
>>>>>>>>>>>>>>>                                                 Engine
>>>>>>>>>>>>>>>                                                 runner
>>>>>>>>>>>>>>>                                                 and the
>>>>>>>>>>>>>>>                                                 DUT
>>>>>>>>>>>>>>>                                                 host)
>>>>>>>>>>>>>>>                                                 are
>>>>>>>>>>>>>>>                                                 running
>>>>>>>>>>>>>>>                                                 Ubuntu
>>>>>>>>>>>>>>>                                                 20.04
>>>>>>>>>>>>>>>                                                 LTS,
>>>>>>>>>>>>>>>                                                 however
>>>>>>>>>>>>>>>                                                 this
>>>>>>>>>>>>>>>                                                 morning
>>>>>>>>>>>>>>>                                                 I
>>>>>>>>>>>>>>>                                                 noticed
>>>>>>>>>>>>>>>                                                 that the
>>>>>>>>>>>>>>>                                                 tester
>>>>>>>>>>>>>>>                                                 system
>>>>>>>>>>>>>>>                                                 (the one
>>>>>>>>>>>>>>>                                                 having
>>>>>>>>>>>>>>>                                                 issues)
>>>>>>>>>>>>>>>                                                 is
>>>>>>>>>>>>>>>                                                 running
>>>>>>>>>>>>>>>                                                 Ubuntu
>>>>>>>>>>>>>>>                                                 22.04 LTS.
>>>>>>>>>>>>>>>                                                 This
>>>>>>>>>>>>>>>                                                 could be
>>>>>>>>>>>>>>>                                                 the
>>>>>>>>>>>>>>>                                                 source
>>>>>>>>>>>>>>>                                                 of the
>>>>>>>>>>>>>>>                                                 problem.
>>>>>>>>>>>>>>>                                                 I
>>>>>>>>>>>>>>>                                                 encountered
>>>>>>>>>>>>>>>                                                 a
>>>>>>>>>>>>>>>                                                 dependency
>>>>>>>>>>>>>>>                                                 issue
>>>>>>>>>>>>>>>                                                 trying
>>>>>>>>>>>>>>>                                                 to run
>>>>>>>>>>>>>>>                                                 the Test
>>>>>>>>>>>>>>>                                                 Engine
>>>>>>>>>>>>>>>                                                 on 22.04
>>>>>>>>>>>>>>>                                                 LTS, so
>>>>>>>>>>>>>>>                                                 I
>>>>>>>>>>>>>>>                                                 downgraded
>>>>>>>>>>>>>>>                                                 the
>>>>>>>>>>>>>>>                                                 system.
>>>>>>>>>>>>>>>                                                 Since
>>>>>>>>>>>>>>>                                                 the
>>>>>>>>>>>>>>>                                                 tester
>>>>>>>>>>>>>>>                                                 is also
>>>>>>>>>>>>>>>                                                 the host
>>>>>>>>>>>>>>>                                                 having
>>>>>>>>>>>>>>>                                                 connection
>>>>>>>>>>>>>>>                                                 issues,
>>>>>>>>>>>>>>>                                                 I will
>>>>>>>>>>>>>>>                                                 try
>>>>>>>>>>>>>>>                                                 downgrading
>>>>>>>>>>>>>>>                                                 that
>>>>>>>>>>>>>>>                                                 system
>>>>>>>>>>>>>>>                                                 to
>>>>>>>>>>>>>>>                                                 20.04,
>>>>>>>>>>>>>>>                                                 and see
>>>>>>>>>>>>>>>                                                 if that
>>>>>>>>>>>>>>>                                                 changes
>>>>>>>>>>>>>>>                                                 anything.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>                                                 Unlikely,
>>>>>>>>>>>>>>                                                 but who
>>>>>>>>>>>>>>                                                 knows. We
>>>>>>>>>>>>>>                                                 run tests
>>>>>>>>>>>>>>                                                 (DUTs) on
>>>>>>>>>>>>>>                                                 Ubuntu
>>>>>>>>>>>>>>                                                 20.04,
>>>>>>>>>>>>>>                                                 Ubuntu
>>>>>>>>>>>>>>                                                 22.04,
>>>>>>>>>>>>>>                                                 Ubuntu
>>>>>>>>>>>>>>                                                 22.10,
>>>>>>>>>>>>>>                                                 Ubuntu
>>>>>>>>>>>>>>                                                 23.04,
>>>>>>>>>>>>>>                                                 Debian 11
>>>>>>>>>>>>>>                                                 and
>>>>>>>>>>>>>>                                                 Fedora 38
>>>>>>>>>>>>>>                                                 every night.
>>>>>>>>>>>>>>                                                 Right now
>>>>>>>>>>>>>>                                                 Debian 11
>>>>>>>>>>>>>>                                                 is used
>>>>>>>>>>>>>>                                                 for test
>>>>>>>>>>>>>>                                                 engine in
>>>>>>>>>>>>>>                                                 nightly
>>>>>>>>>>>>>>                                                 regressions.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>                                                 I'll
>>>>>>>>>>>>>>                                                 double-check
>>>>>>>>>>>>>>                                                 test
>>>>>>>>>>>>>>                                                 engine on
>>>>>>>>>>>>>>                                                 Ubuntu
>>>>>>>>>>>>>>                                                 20.04 and
>>>>>>>>>>>>>>                                                 Ubuntu 22.04.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>                                                 I did
>>>>>>>>>>>>>>>                                                 try
>>>>>>>>>>>>>>>                                                 passing
>>>>>>>>>>>>>>>                                                 in the
>>>>>>>>>>>>>>>                                                 "--vg-rcf"
>>>>>>>>>>>>>>>                                                 argument
>>>>>>>>>>>>>>>                                                 to the
>>>>>>>>>>>>>>>                                                 run.sh
>>>>>>>>>>>>>>>                                                 script
>>>>>>>>>>>>>>>                                                 of the
>>>>>>>>>>>>>>>                                                 test
>>>>>>>>>>>>>>>                                                 suite
>>>>>>>>>>>>>>>                                                 after
>>>>>>>>>>>>>>>                                                 installing
>>>>>>>>>>>>>>>                                                 valgrind,
>>>>>>>>>>>>>>>                                                 but
>>>>>>>>>>>>>>>                                                 there
>>>>>>>>>>>>>>>                                                 was no
>>>>>>>>>>>>>>>                                                 additional
>>>>>>>>>>>>>>>                                                 output
>>>>>>>>>>>>>>>                                                 that I saw.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>                                                 Sorry, I
>>>>>>>>>>>>>>                                                 should
>>>>>>>>>>>>>>                                                 valgrind
>>>>>>>>>>>>>>                                                 output
>>>>>>>>>>>>>>                                                 should be
>>>>>>>>>>>>>>                                                 in
>>>>>>>>>>>>>>                                                 valgrind.te_rcf
>>>>>>>>>>>>>>                                                 (direction
>>>>>>>>>>>>>>                                                 where you
>>>>>>>>>>>>>>                                                 run test
>>>>>>>>>>>>>>                                                 engine).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>                                                 I will
>>>>>>>>>>>>>>>                                                 try
>>>>>>>>>>>>>>>                                                 pulling
>>>>>>>>>>>>>>>                                                 in the
>>>>>>>>>>>>>>>                                                 changes
>>>>>>>>>>>>>>>                                                 you've
>>>>>>>>>>>>>>>                                                 pushed
>>>>>>>>>>>>>>>                                                 up, and
>>>>>>>>>>>>>>>                                                 will see
>>>>>>>>>>>>>>>                                                 if that
>>>>>>>>>>>>>>>                                                 fixes
>>>>>>>>>>>>>>>                                                 anything.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>                                                 Thanks,
>>>>>>>>>>>>>>>                                                 Adam
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>                                                 On Fri,
>>>>>>>>>>>>>>>                                                 Aug 25,
>>>>>>>>>>>>>>>                                                 2023 at
>>>>>>>>>>>>>>>                                                 9:57 AM
>>>>>>>>>>>>>>>                                                 Andrew
>>>>>>>>>>>>>>>                                                 Rybchenko
>>>>>>>>>>>>>>>                                                 <andrew.rybchenko@oktetlabs.ru>
>>>>>>>>>>>>>>>                                                 wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>                                                     Hello
>>>>>>>>>>>>>>>                                                     Adam,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>                                                     On
>>>>>>>>>>>>>>>                                                     8/24/23
>>>>>>>>>>>>>>>                                                     23:54,
>>>>>>>>>>>>>>>                                                     Andrew
>>>>>>>>>>>>>>>                                                     Rybchenko
>>>>>>>>>>>>>>>                                                     wrote:
>>>>>>>>>>>>>>>>                                                     I'd
>>>>>>>>>>>>>>>>                                                     like
>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>                                                     try
>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>                                                     repeat
>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>                                                     problem
>>>>>>>>>>>>>>>>                                                     locally.
>>>>>>>>>>>>>>>>                                                     Which
>>>>>>>>>>>>>>>>                                                     Linux
>>>>>>>>>>>>>>>>                                                     distro
>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>                                                     running
>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>                                                     engine
>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>                                                     agents?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>                                                     In
>>>>>>>>>>>>>>>>                                                     fact
>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>                                                     know
>>>>>>>>>>>>>>>>                                                     one
>>>>>>>>>>>>>>>>                                                     problem
>>>>>>>>>>>>>>>>                                                     with
>>>>>>>>>>>>>>>>                                                     Debian
>>>>>>>>>>>>>>>>                                                     12
>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>                                                     Fedora
>>>>>>>>>>>>>>>>                                                     38
>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>                                                     we have
>>>>>>>>>>>>>>>>                                                     patch
>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>                                                     review
>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>                                                     fix
>>>>>>>>>>>>>>>>                                                     it,
>>>>>>>>>>>>>>>>                                                     however,
>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>                                                     behaviour
>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>                                                     different
>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>                                                     this
>>>>>>>>>>>>>>>>                                                     case,
>>>>>>>>>>>>>>>>                                                     so
>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>                                                     unlike
>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>                                                     same
>>>>>>>>>>>>>>>>                                                     problem.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>                                                     I've
>>>>>>>>>>>>>>>                                                     just
>>>>>>>>>>>>>>>                                                     published
>>>>>>>>>>>>>>>                                                     a
>>>>>>>>>>>>>>>                                                     new
>>>>>>>>>>>>>>>                                                     tag
>>>>>>>>>>>>>>>                                                     which
>>>>>>>>>>>>>>>                                                     fixes
>>>>>>>>>>>>>>>                                                     known
>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>                                                     engine
>>>>>>>>>>>>>>>                                                     side
>>>>>>>>>>>>>>>                                                     problems
>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>                                                     Debian
>>>>>>>>>>>>>>>                                                     12
>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>                                                     Fedora
>>>>>>>>>>>>>>>                                                     38.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>                                                     One
>>>>>>>>>>>>>>>>                                                     more
>>>>>>>>>>>>>>>>                                                     idea
>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>                                                     install
>>>>>>>>>>>>>>>>                                                     valgrind
>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>                                                     engine
>>>>>>>>>>>>>>>>                                                     host
>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>                                                     run
>>>>>>>>>>>>>>>>                                                     with
>>>>>>>>>>>>>>>>                                                     option
>>>>>>>>>>>>>>>>                                                     --vg-rcf
>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>                                                     check
>>>>>>>>>>>>>>>>                                                     if
>>>>>>>>>>>>>>>>                                                     something
>>>>>>>>>>>>>>>>                                                     weird
>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>                                                     happening.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>                                                     What
>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>                                                     don't
>>>>>>>>>>>>>>>>                                                     understand
>>>>>>>>>>>>>>>>                                                     right
>>>>>>>>>>>>>>>>                                                     now
>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>                                                     why
>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>                                                     see
>>>>>>>>>>>>>>>>                                                     just
>>>>>>>>>>>>>>>>                                                     one
>>>>>>>>>>>>>>>>                                                     failed
>>>>>>>>>>>>>>>>                                                     attempt
>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>                                                     connect
>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>                                                     your
>>>>>>>>>>>>>>>>                                                     log.txt
>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>                                                     then
>>>>>>>>>>>>>>>>                                                     Logger
>>>>>>>>>>>>>>>>                                                     shutdown
>>>>>>>>>>>>>>>>                                                     after
>>>>>>>>>>>>>>>>                                                     9
>>>>>>>>>>>>>>>>                                                     minutes.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>                                                     Andrew.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>                                                     On
>>>>>>>>>>>>>>>>                                                     8/24/23
>>>>>>>>>>>>>>>>                                                     23:29,
>>>>>>>>>>>>>>>>                                                     Adam
>>>>>>>>>>>>>>>>                                                     Hassick
>>>>>>>>>>>>>>>>                                                     wrote:
>>>>>>>>>>>>>>>>>                                                      >
>>>>>>>>>>>>>>>>>                                                     Is
>>>>>>>>>>>>>>>>>                                                     there
>>>>>>>>>>>>>>>>>                                                     any
>>>>>>>>>>>>>>>>>                                                     firewall
>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     network
>>>>>>>>>>>>>>>>>                                                     or
>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>                                                     hosts
>>>>>>>>>>>>>>>>>                                                     which
>>>>>>>>>>>>>>>>>                                                     could
>>>>>>>>>>>>>>>>>                                                     block
>>>>>>>>>>>>>>>>>                                                     incoming
>>>>>>>>>>>>>>>>>                                                     TCP
>>>>>>>>>>>>>>>>>                                                     connection
>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     port
>>>>>>>>>>>>>>>>>                                                     23571
>>>>>>>>>>>>>>>>>                                                     <http://iol-dts-tester.dpdklab.iol.unh.edu:23571>
>>>>>>>>>>>>>>>>>                                                     <http://iol-dts-tester.dpdklab.iol.unh.edu:23571>
>>>>>>>>>>>>>>>>>                                                     from
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     host
>>>>>>>>>>>>>>>>>                                                     where
>>>>>>>>>>>>>>>>>                                                     you
>>>>>>>>>>>>>>>>>                                                     run
>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>                                                     engine?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                     Our
>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>                                                     engine
>>>>>>>>>>>>>>>>>                                                     host
>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     testbed
>>>>>>>>>>>>>>>>>                                                     are
>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     same
>>>>>>>>>>>>>>>>>                                                     subnet.
>>>>>>>>>>>>>>>>>                                                     The
>>>>>>>>>>>>>>>>>                                                     connection
>>>>>>>>>>>>>>>>>                                                     does
>>>>>>>>>>>>>>>>>                                                     work
>>>>>>>>>>>>>>>>>                                                     sometimes.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                      >
>>>>>>>>>>>>>>>>>                                                     If
>>>>>>>>>>>>>>>>>                                                     behaviour
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     same
>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     next
>>>>>>>>>>>>>>>>>                                                     try
>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>                                                     you
>>>>>>>>>>>>>>>>>                                                     see
>>>>>>>>>>>>>>>>>                                                     that
>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>                                                     agent
>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>                                                     kept
>>>>>>>>>>>>>>>>>                                                     running,
>>>>>>>>>>>>>>>>>                                                     could
>>>>>>>>>>>>>>>>>                                                     you
>>>>>>>>>>>>>>>>>                                                     check
>>>>>>>>>>>>>>>>>                                                     using
>>>>>>>>>>>>>>>>>                                                      >
>>>>>>>>>>>>>>>>>                                                      >
>>>>>>>>>>>>>>>>>                                                     #
>>>>>>>>>>>>>>>>>                                                     netstat
>>>>>>>>>>>>>>>>>                                                     -tnlp
>>>>>>>>>>>>>>>>>                                                      >
>>>>>>>>>>>>>>>>>                                                      >
>>>>>>>>>>>>>>>>>                                                     that
>>>>>>>>>>>>>>>>>                                                     Test
>>>>>>>>>>>>>>>>>                                                     Agent
>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>                                                     listening
>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     port
>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>                                                     try
>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>                                                     establish
>>>>>>>>>>>>>>>>>                                                     TCP
>>>>>>>>>>>>>>>>>                                                     connection
>>>>>>>>>>>>>>>>>                                                     from
>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>                                                     agent
>>>>>>>>>>>>>>>>>                                                     using
>>>>>>>>>>>>>>>>>                                                      >
>>>>>>>>>>>>>>>>>                                                      >
>>>>>>>>>>>>>>>>>                                                     $
>>>>>>>>>>>>>>>>>                                                     telnet
>>>>>>>>>>>>>>>>>                                                     iol-dts-tester.dpdklab.iol.unh.edu
>>>>>>>>>>>>>>>>>                                                     <http://iol-dts-tester.dpdklab.iol.unh.edu>
>>>>>>>>>>>>>>>>>                                                     <http://iol-dts-tester.dpdklab.iol.unh.edu:23571>
>>>>>>>>>>>>>>>>>                                                     <http://iol-dts-tester.dpdklab.iol.unh.edu:23571>
>>>>>>>>>>>>>>>>>                                                     23571
>>>>>>>>>>>>>>>>>                                                     <http://iol-dts-tester.dpdklab.iol.unh.edu:23571>
>>>>>>>>>>>>>>>>>                                                     <http://iol-dts-tester.dpdklab.iol.unh.edu:23571>
>>>>>>>>>>>>>>>>>                                                      >
>>>>>>>>>>>>>>>>>                                                      >
>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>                                                     check
>>>>>>>>>>>>>>>>>                                                     if
>>>>>>>>>>>>>>>>>                                                     TCP
>>>>>>>>>>>>>>>>>                                                     connection
>>>>>>>>>>>>>>>>>                                                     could
>>>>>>>>>>>>>>>>>                                                     be
>>>>>>>>>>>>>>>>>                                                     established.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>                                                     was
>>>>>>>>>>>>>>>>>                                                     able
>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>                                                     replicate
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     same
>>>>>>>>>>>>>>>>>                                                     behavior
>>>>>>>>>>>>>>>>>                                                     again,
>>>>>>>>>>>>>>>>>                                                     where
>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>                                                     hangs
>>>>>>>>>>>>>>>>>                                                     while
>>>>>>>>>>>>>>>>>                                                     RCF
>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>                                                     trying
>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>                                                     start.
>>>>>>>>>>>>>>>>>                                                     Running
>>>>>>>>>>>>>>>>>                                                     this
>>>>>>>>>>>>>>>>>                                                     command,
>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>                                                     see
>>>>>>>>>>>>>>>>>                                                     this
>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     output:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                     tcp
>>>>>>>>>>>>>>>>>                                                      
>>>>>>>>>>>>>>>>>                                                      
>>>>>>>>>>>>>>>>>                                                      
>>>>>>>>>>>>>>>>>                                                      0
>>>>>>>>>>>>>>>>>                                                      
>>>>>>>>>>>>>>>>>                                                      0
>>>>>>>>>>>>>>>>>                                                     0.0.0.0:23571
>>>>>>>>>>>>>>>>>                                                     <http://0.0.0.0:23571>
>>>>>>>>>>>>>>>>>                                                     <http://0.0.0.0:23571>
>>>>>>>>>>>>>>>>>                                                     <http://0.0.0.0:23571>
>>>>>>>>>>>>>>>>>                                                     0.0.0.0:*
>>>>>>>>>>>>>>>>>                                                     LISTEN
>>>>>>>>>>>>>>>>>                                                      18599/ta
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                     So
>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>                                                     seems
>>>>>>>>>>>>>>>>>                                                     like
>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>                                                     listening
>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     correct
>>>>>>>>>>>>>>>>>                                                     port.
>>>>>>>>>>>>>>>>>                                                     Additionally,
>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>                                                     was
>>>>>>>>>>>>>>>>>                                                     able
>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>                                                     connect
>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     Tester
>>>>>>>>>>>>>>>>>                                                     machine
>>>>>>>>>>>>>>>>>                                                     from
>>>>>>>>>>>>>>>>>                                                     our
>>>>>>>>>>>>>>>>>                                                     Test
>>>>>>>>>>>>>>>>>                                                     Engine
>>>>>>>>>>>>>>>>>                                                     host
>>>>>>>>>>>>>>>>>                                                     using
>>>>>>>>>>>>>>>>>                                                     telnet.
>>>>>>>>>>>>>>>>>                                                     It
>>>>>>>>>>>>>>>>>                                                     printed
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     PID
>>>>>>>>>>>>>>>>>                                                     of
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     process
>>>>>>>>>>>>>>>>>                                                     once
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     connection
>>>>>>>>>>>>>>>>>                                                     was
>>>>>>>>>>>>>>>>>                                                     opened.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>                                                     tried
>>>>>>>>>>>>>>>>>                                                     running
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     "ta"
>>>>>>>>>>>>>>>>>                                                     application
>>>>>>>>>>>>>>>>>                                                     manually
>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     command
>>>>>>>>>>>>>>>>>                                                     line,
>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>                                                     didn't
>>>>>>>>>>>>>>>>>                                                     print
>>>>>>>>>>>>>>>>>                                                     anything
>>>>>>>>>>>>>>>>>                                                     at
>>>>>>>>>>>>>>>>>                                                     all.
>>>>>>>>>>>>>>>>>                                                     Maybe
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     issue
>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>                                                     something
>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     Test
>>>>>>>>>>>>>>>>>                                                     Engine
>>>>>>>>>>>>>>>>>                                                     side.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                     On
>>>>>>>>>>>>>>>>>                                                     Thu,
>>>>>>>>>>>>>>>>>                                                     Aug
>>>>>>>>>>>>>>>>>                                                     24,
>>>>>>>>>>>>>>>>>                                                     2023
>>>>>>>>>>>>>>>>>                                                     at
>>>>>>>>>>>>>>>>>                                                     2:35 PM
>>>>>>>>>>>>>>>>>                                                     Andrew
>>>>>>>>>>>>>>>>>                                                     Rybchenko
>>>>>>>>>>>>>>>>>                                                     <andrew.rybchenko@oktetlabs.ru
>>>>>>>>>>>>>>>>>                                                     <mailto:andrew.rybchenko@oktetlabs.ru>
>>>>>>>>>>>>>>>>>                                                     <mailto:andrew.rybchenko@oktetlabs.ru>>
>>>>>>>>>>>>>>>>>                                                     wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     Hi
>>>>>>>>>>>>>>>>>                                                     Adam,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                         
>>>>>>>>>>>>>>>>>                                                     >
>>>>>>>>>>>>>>>>>                                                     On
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     tester
>>>>>>>>>>>>>>>>>                                                     host
>>>>>>>>>>>>>>>>>                                                     (which
>>>>>>>>>>>>>>>>>                                                     appears
>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>                                                     be
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     Peer
>>>>>>>>>>>>>>>>>                                                     agent),
>>>>>>>>>>>>>>>>>                                                     there
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     are
>>>>>>>>>>>>>>>>>                                                     four
>>>>>>>>>>>>>>>>>                                                     processes
>>>>>>>>>>>>>>>>>                                                     that
>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>                                                     see
>>>>>>>>>>>>>>>>>                                                     running,
>>>>>>>>>>>>>>>>>                                                     which
>>>>>>>>>>>>>>>>>                                                     look
>>>>>>>>>>>>>>>>>                                                     like
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     agent
>>>>>>>>>>>>>>>>>                                                     processes.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     Before
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     next
>>>>>>>>>>>>>>>>>                                                     try
>>>>>>>>>>>>>>>>>                                                     I'd
>>>>>>>>>>>>>>>>>                                                     recommend
>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>                                                     kill
>>>>>>>>>>>>>>>>>                                                     these
>>>>>>>>>>>>>>>>>                                                     processes.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     Is
>>>>>>>>>>>>>>>>>                                                     there
>>>>>>>>>>>>>>>>>                                                     any
>>>>>>>>>>>>>>>>>                                                     firewall
>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     network
>>>>>>>>>>>>>>>>>                                                     or
>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>                                                     hosts
>>>>>>>>>>>>>>>>>                                                     which
>>>>>>>>>>>>>>>>>                                                     could
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     block
>>>>>>>>>>>>>>>>>                                                     incoming
>>>>>>>>>>>>>>>>>                                                     TCP
>>>>>>>>>>>>>>>>>                                                     connection
>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     port
>>>>>>>>>>>>>>>>>                                                     23571
>>>>>>>>>>>>>>>>>                                                     <http://iol-dts-tester.dpdklab.iol.unh.edu:23571>
>>>>>>>>>>>>>>>>>                                                     <http://iol-dts-tester.dpdklab.iol.unh.edu:23571>
>>>>>>>>>>>>>>>>>                                                     from
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     host
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     where
>>>>>>>>>>>>>>>>>                                                     you
>>>>>>>>>>>>>>>>>                                                     run
>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>                                                     engine?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     If
>>>>>>>>>>>>>>>>>                                                     behaviour
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     same
>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     next
>>>>>>>>>>>>>>>>>                                                     try
>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>                                                     you
>>>>>>>>>>>>>>>>>                                                     see
>>>>>>>>>>>>>>>>>                                                     that
>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>                                                     agent
>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     kept
>>>>>>>>>>>>>>>>>                                                     running,
>>>>>>>>>>>>>>>>>                                                     could
>>>>>>>>>>>>>>>>>                                                     you
>>>>>>>>>>>>>>>>>                                                     check
>>>>>>>>>>>>>>>>>                                                     using
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     #
>>>>>>>>>>>>>>>>>                                                     netstat
>>>>>>>>>>>>>>>>>                                                     -tnlp
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     that
>>>>>>>>>>>>>>>>>                                                     Test
>>>>>>>>>>>>>>>>>                                                     Agent
>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>                                                     listening
>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     port
>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>                                                     try
>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>                                                     establish
>>>>>>>>>>>>>>>>>                                                     TCP
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     connection
>>>>>>>>>>>>>>>>>                                                     from
>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>                                                     agent
>>>>>>>>>>>>>>>>>                                                     using
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     $
>>>>>>>>>>>>>>>>>                                                     telnet
>>>>>>>>>>>>>>>>>                                                     iol-dts-tester.dpdklab.iol.unh.edu
>>>>>>>>>>>>>>>>>                                                     <http://iol-dts-tester.dpdklab.iol.unh.edu>
>>>>>>>>>>>>>>>>>                                                     <http://iol-dts-tester.dpdklab.iol.unh.edu:23571>
>>>>>>>>>>>>>>>>>                                                     <http://iol-dts-tester.dpdklab.iol.unh.edu:23571>
>>>>>>>>>>>>>>>>>                                                     23571
>>>>>>>>>>>>>>>>>                                                     <http://iol-dts-tester.dpdklab.iol.unh.edu:23571>
>>>>>>>>>>>>>>>>>                                                     <http://iol-dts-tester.dpdklab.iol.unh.edu:23571>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>                                                     check
>>>>>>>>>>>>>>>>>                                                     if
>>>>>>>>>>>>>>>>>                                                     TCP
>>>>>>>>>>>>>>>>>                                                     connection
>>>>>>>>>>>>>>>>>                                                     could
>>>>>>>>>>>>>>>>>                                                     be
>>>>>>>>>>>>>>>>>                                                     established.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     Another
>>>>>>>>>>>>>>>>>                                                     idea
>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>                                                     login
>>>>>>>>>>>>>>>>>                                                     Tester
>>>>>>>>>>>>>>>>>                                                     under
>>>>>>>>>>>>>>>>>                                                     root
>>>>>>>>>>>>>>>>>                                                     as
>>>>>>>>>>>>>>>>>                                                     testing
>>>>>>>>>>>>>>>>>                                                     does,
>>>>>>>>>>>>>>>>>                                                     get
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     start
>>>>>>>>>>>>>>>>>                                                     TA
>>>>>>>>>>>>>>>>>                                                     command
>>>>>>>>>>>>>>>>>                                                     from
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     log
>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>                                                     try
>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>                                                     by
>>>>>>>>>>>>>>>>>                                                     hands
>>>>>>>>>>>>>>>>>                                                     without
>>>>>>>>>>>>>>>>>                                                     -n and
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     remove
>>>>>>>>>>>>>>>>>                                                     extra
>>>>>>>>>>>>>>>>>                                                     escaping.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     #
>>>>>>>>>>>>>>>>>                                                     sudo
>>>>>>>>>>>>>>>>>                                                     PATH=${PATH}:/tmp/linux_x86_root_76872_1692885663_1
>>>>>>>>>>>>>>>>>                                                     LD_LIBRARY_PATH=${LD_LIBRARY_PATH}${LD_LIBRARY_PATH:+:}/tmp/linux_x86_root_76872_1692885663_1
>>>>>>>>>>>>>>>>>                                                     /tmp/linux_x86_root_76872_1692885663_1/ta
>>>>>>>>>>>>>>>>>                                                     Peer
>>>>>>>>>>>>>>>>>                                                     23571
>>>>>>>>>>>>>>>>>                                                     host=iol-dts-tester.dpdklab.iol.unh.edu:port=23571:user=root:key=/opt/tsf/keys/id_ed25519:ssh_port=22:copy_timeout=15:kill_timeout=15:sudo=:shell=
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     Hopefully
>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>                                                     this
>>>>>>>>>>>>>>>>>                                                     case
>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>                                                     agent
>>>>>>>>>>>>>>>>>                                                     directory
>>>>>>>>>>>>>>>>>                                                     remains
>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>                                                     /tmp
>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     you
>>>>>>>>>>>>>>>>>                                                     don't
>>>>>>>>>>>>>>>>>                                                     need
>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>                                                     copy
>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>                                                     as
>>>>>>>>>>>>>>>>>                                                     testing
>>>>>>>>>>>>>>>>>                                                     does.
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     May
>>>>>>>>>>>>>>>>>                                                     be
>>>>>>>>>>>>>>>>>                                                     output
>>>>>>>>>>>>>>>>>                                                     could
>>>>>>>>>>>>>>>>>                                                     shed
>>>>>>>>>>>>>>>>>                                                     some
>>>>>>>>>>>>>>>>>                                                     light
>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>                                                     what's
>>>>>>>>>>>>>>>>>                                                     going
>>>>>>>>>>>>>>>>>                                                     on.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     Andrew.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>                                                     On
>>>>>>>>>>>>>>>>>                                                     8/24/23
>>>>>>>>>>>>>>>>>                                                     17:30,
>>>>>>>>>>>>>>>>>                                                     Adam
>>>>>>>>>>>>>>>>>                                                     Hassick
>>>>>>>>>>>>>>>>>                                                     wrote:
>>>>>>>>>>>>>>>>>>                                                     Hi
>>>>>>>>>>>>>>>>>>                                                     Andrew,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>>                                                     This
>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>                                                     output
>>>>>>>>>>>>>>>>>>                                                     that
>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>                                                     see
>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>                                                     terminal
>>>>>>>>>>>>>>>>>>                                                     when
>>>>>>>>>>>>>>>>>>                                                     this
>>>>>>>>>>>>>>>>>>                                                     failure
>>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>>                                                     occurs,
>>>>>>>>>>>>>>>>>>                                                     after
>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>                                                     agent
>>>>>>>>>>>>>>>>>>                                                     binaries
>>>>>>>>>>>>>>>>>>                                                     build
>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>                                                     engine
>>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>>                                                     starts:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>>                                                     Platform
>>>>>>>>>>>>>>>>>>                                                     default
>>>>>>>>>>>>>>>>>>                                                     build
>>>>>>>>>>>>>>>>>>                                                     -
>>>>>>>>>>>>>>>>>>                                                     pass
>>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>>                                                     Simple
>>>>>>>>>>>>>>>>>>                                                     RCF
>>>>>>>>>>>>>>>>>>                                                     consistency
>>>>>>>>>>>>>>>>>>                                                     check
>>>>>>>>>>>>>>>>>>                                                     succeeded
>>>>>>>>>>>>>>>>>>                                                     --->>>
>>>>>>>>>>>>>>>>>>                                                     Starting
>>>>>>>>>>>>>>>>>>                                                     Logger...done
>>>>>>>>>>>>>>>>>>                                                     --->>>
>>>>>>>>>>>>>>>>>>                                                     Starting
>>>>>>>>>>>>>>>>>>                                                     RCF...rcf_net_engine_connect():
>>>>>>>>>>>>>>>>>>                                                     Connection
>>>>>>>>>>>>>>>>>>                                                     timed
>>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>>                                                     out
>>>>>>>>>>>>>>>>>>                                                     iol-dts-tester.dpdklab.iol.unh.edu:23571
>>>>>>>>>>>>>>>>>>                                                     <http://iol-dts-tester.dpdklab.iol.unh.edu:23571>
>>>>>>>>>>>>>>>>>>                                                     <http://iol-dts-tester.dpdklab.iol.unh.edu:23571>
>>>>>>>>>>>>>>>>>>                                                     <http://iol-dts-tester.dpdklab.iol.unh.edu:23571>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>>                                                     Then,
>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>                                                     hangs
>>>>>>>>>>>>>>>>>>                                                     here
>>>>>>>>>>>>>>>>>>                                                     until
>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>                                                     kill
>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>                                                     "te_rcf"
>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>                                                     "te_tee"
>>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>>                                                     processes.
>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>                                                     let
>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>                                                     hang
>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>                                                     around
>>>>>>>>>>>>>>>>>>                                                     9
>>>>>>>>>>>>>>>>>>                                                     minutes.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>>                                                     On
>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>                                                     tester
>>>>>>>>>>>>>>>>>>                                                     host
>>>>>>>>>>>>>>>>>>                                                     (which
>>>>>>>>>>>>>>>>>>                                                     appears
>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>                                                     be
>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>                                                     Peer
>>>>>>>>>>>>>>>>>>                                                     agent),
>>>>>>>>>>>>>>>>>>                                                     there
>>>>>>>>>>>>>>>>>>                                                     are
>>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>>                                                     four
>>>>>>>>>>>>>>>>>>                                                     processes
>>>>>>>>>>>>>>>>>>                                                     that
>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>                                                     see
>>>>>>>>>>>>>>>>>>                                                     running,
>>>>>>>>>>>>>>>>>>                                                     which
>>>>>>>>>>>>>>>>>>                                                     look
>>>>>>>>>>>>>>>>>>                                                     like
>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>                                                     agent
>>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>>                                                     processes.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>>                                                     ta.Peer
>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>                                                     an
>>>>>>>>>>>>>>>>>>                                                     empty
>>>>>>>>>>>>>>>>>>                                                     file.
>>>>>>>>>>>>>>>>>>                                                     I've
>>>>>>>>>>>>>>>>>>                                                     attached
>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>                                                     log.txt
>>>>>>>>>>>>>>>>>>                                                     from
>>>>>>>>>>>>>>>>>>                                                     this
>>>>>>>>>>>>>>>>>>                                                     run.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>>                                                      -
>>>>>>>>>>>>>>>>>>                                                     Adam
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>>                                                     On
>>>>>>>>>>>>>>>>>>                                                     Thu,
>>>>>>>>>>>>>>>>>>                                                     Aug
>>>>>>>>>>>>>>>>>>                                                     24,
>>>>>>>>>>>>>>>>>>                                                     2023
>>>>>>>>>>>>>>>>>>                                                     at
>>>>>>>>>>>>>>>>>>                                                     4:22��AM
>>>>>>>>>>>>>>>>>>                                                     Andrew
>>>>>>>>>>>>>>>>>>                                                     Rybchenko
>>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>>                                                     <andrew.rybchenko@oktetlabs.ru
>>>>>>>>>>>>>>>>>>                                                     <mailto:andrew.rybchenko@oktetlabs.ru>
>>>>>>>>>>>>>>>>>>                                                     <mailto:andrew.rybchenko@oktetlabs.ru>>
>>>>>>>>>>>>>>>>>>                                                     wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>                                                     Hi
>>>>>>>>>>>>>>>>>>                                                     Adam,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>                                                     Yes,
>>>>>>>>>>>>>>>>>>                                                     TE_RCFUNIX_TIMEOUT
>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>                                                     seconds.
>>>>>>>>>>>>>>>>>>                                                     I've
>>>>>>>>>>>>>>>>>>                                                     double-checked
>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>                                                     that
>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>                                                     goes
>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>                                                     'copy_timeout'
>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>                                                     ts-conf/rcf.conf.
>>>>>>>>>>>>>>>>>>                                                     Description
>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>                                                     doc/sphinx/pages/group_te_engine_rcf.rst
>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>                                                     says
>>>>>>>>>>>>>>>>>>                                                     that
>>>>>>>>>>>>>>>>>>                                                     copy_timeout
>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>                                                     seconds
>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>                                                     implementation
>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>                                                     lib/rcfunix/rcfunix.c
>>>>>>>>>>>>>>>>>>                                                     passes
>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>                                                     value
>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>                                                     select()
>>>>>>>>>>>>>>>>>>                                                     tv_sec.
>>>>>>>>>>>>>>>>>>                                                     Theoretically
>>>>>>>>>>>>>>>>>>                                                     select()
>>>>>>>>>>>>>>>>>>                                                     could
>>>>>>>>>>>>>>>>>>                                                     be
>>>>>>>>>>>>>>>>>>                                                     interrupted
>>>>>>>>>>>>>>>>>>                                                     by
>>>>>>>>>>>>>>>>>>                                                     signal,
>>>>>>>>>>>>>>>>>>                                                     but I
>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>                                                     think
>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>                                                     unlikely
>>>>>>>>>>>>>>>>>>                                                     here.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>                                                     I'm
>>>>>>>>>>>>>>>>>>                                                     not
>>>>>>>>>>>>>>>>>>                                                     sure
>>>>>>>>>>>>>>>>>>                                                     that
>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>                                                     understand
>>>>>>>>>>>>>>>>>>                                                     what
>>>>>>>>>>>>>>>>>>                                                     do
>>>>>>>>>>>>>>>>>>                                                     you
>>>>>>>>>>>>>>>>>>                                                     mean
>>>>>>>>>>>>>>>>>>                                                     by
>>>>>>>>>>>>>>>>>>                                                     RCF
>>>>>>>>>>>>>>>>>>                                                     connection
>>>>>>>>>>>>>>>>>>                                                     timeout.
>>>>>>>>>>>>>>>>>>                                                     Does
>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>                                                     happen
>>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>>                                                     TE
>>>>>>>>>>>>>>>>>>                                                     startup
>>>>>>>>>>>>>>>>>>                                                     when
>>>>>>>>>>>>>>>>>>                                                     RCF
>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>                                                     starts
>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>                                                     agents.
>>>>>>>>>>>>>>>>>>                                                     If
>>>>>>>>>>>>>>>>>>                                                     so,
>>>>>>>>>>>>>>>>>>                                                     TE_RCFUNIX_TIMEOUT
>>>>>>>>>>>>>>>>>>                                                     could
>>>>>>>>>>>>>>>>>>                                                     help.
>>>>>>>>>>>>>>>>>>                                                     Or
>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>                                                     does
>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>                                                     happen
>>>>>>>>>>>>>>>>>>                                                     when
>>>>>>>>>>>>>>>>>>                                                     tests
>>>>>>>>>>>>>>>>>>                                                     are
>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>                                                     progress,
>>>>>>>>>>>>>>>>>>                                                     e.g.
>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>                                                     middle
>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>                                                     of
>>>>>>>>>>>>>>>>>>                                                     a
>>>>>>>>>>>>>>>>>>                                                     test.
>>>>>>>>>>>>>>>>>>                                                     If
>>>>>>>>>>>>>>>>>>                                                     so,
>>>>>>>>>>>>>>>>>>                                                     TE_RCFUNIX_TIMEOUT
>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>                                                     unrelated
>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>                                                     most
>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>                                                     likely
>>>>>>>>>>>>>>>>>>                                                     either
>>>>>>>>>>>>>>>>>>                                                     host
>>>>>>>>>>>>>>>>>>                                                     with
>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>                                                     agent
>>>>>>>>>>>>>>>>>>                                                     dies
>>>>>>>>>>>>>>>>>>                                                     or
>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>                                                     agent
>>>>>>>>>>>>>>>>>>                                                     itself
>>>>>>>>>>>>>>>>>>                                                     crashes.
>>>>>>>>>>>>>>>>>>                                                     It
>>>>>>>>>>>>>>>>>>                                                     would
>>>>>>>>>>>>>>>>>>                                                     be
>>>>>>>>>>>>>>>>>>                                                     easier
>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>                                                     me
>>>>>>>>>>>>>>>>>>                                                     if
>>>>>>>>>>>>>>>>>>                                                     classify
>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>                                                     if
>>>>>>>>>>>>>>>>>>                                                     you
>>>>>>>>>>>>>>>>>>                                                     share
>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>                                                     text
>>>>>>>>>>>>>>>>>>                                                     log
>>>>>>>>>>>>>>>>>>                                                     (log.txt,
>>>>>>>>>>>>>>>>>>                                                     full
>>>>>>>>>>>>>>>>>>                                                     or
>>>>>>>>>>>>>>>>>>                                                     just
>>>>>>>>>>>>>>>>>>                                                     corresponding
>>>>>>>>>>>>>>>>>>                                                     fragment
>>>>>>>>>>>>>>>>>>                                                     with
>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>                                                     some
>>>>>>>>>>>>>>>>>>                                                     context).
>>>>>>>>>>>>>>>>>>                                                     Also
>>>>>>>>>>>>>>>>>>                                                     content
>>>>>>>>>>>>>>>>>>                                                     of
>>>>>>>>>>>>>>>>>>                                                     ta.DPDK
>>>>>>>>>>>>>>>>>>                                                     or
>>>>>>>>>>>>>>>>>>                                                     ta.Peer
>>>>>>>>>>>>>>>>>>                                                     file
>>>>>>>>>>>>>>>>>>                                                     depending
>>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>>                                                     which
>>>>>>>>>>>>>>>>>>                                                     agent
>>>>>>>>>>>>>>>>>>                                                     has
>>>>>>>>>>>>>>>>>>                                                     problems
>>>>>>>>>>>>>>>>>>                                                     could
>>>>>>>>>>>>>>>>>>                                                     shed
>>>>>>>>>>>>>>>>>>                                                     some
>>>>>>>>>>>>>>>>>>                                                     light.
>>>>>>>>>>>>>>>>>>                                                     Corresponding
>>>>>>>>>>>>>>>>>>                                                     files
>>>>>>>>>>>>>>>>>>                                                     contain
>>>>>>>>>>>>>>>>>>                                                     stdout/stderr
>>>>>>>>>>>>>>>>>>                                                     of
>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>                                                     agents.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>                                                     Andrew.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>                                                     On
>>>>>>>>>>>>>>>>>>                                                     8/23/23
>>>>>>>>>>>>>>>>>>                                                     17:45,
>>>>>>>>>>>>>>>>>>                                                     Adam
>>>>>>>>>>>>>>>>>>                                                     Hassick
>>>>>>>>>>>>>>>>>>                                                     wrote:
>>>>>>>>>>>>>>>>>>>                                                     Hi
>>>>>>>>>>>>>>>>>>>                                                     Andrew,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>>                                                     I've
>>>>>>>>>>>>>>>>>>>                                                     set
>>>>>>>>>>>>>>>>>>>                                                     up
>>>>>>>>>>>>>>>>>>>                                                     a
>>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>>                                                     rig
>>>>>>>>>>>>>>>>>>>                                                     repository
>>>>>>>>>>>>>>>>>>>                                                     here,
>>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>>                                                     have
>>>>>>>>>>>>>>>>>>>                                                     created
>>>>>>>>>>>>>>>>>>>                                                     configurations
>>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>>                                                     our
>>>>>>>>>>>>>>>>>>>                                                     development
>>>>>>>>>>>>>>>>>>>                                                     testbed
>>>>>>>>>>>>>>>>>>>                                                     based
>>>>>>>>>>>>>>>>>>>                                                     off
>>>>>>>>>>>>>>>>>>>                                                     of
>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>                                                     examples.
>>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>>                                                     We've
>>>>>>>>>>>>>>>>>>>                                                     been
>>>>>>>>>>>>>>>>>>>                                                     able
>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>                                                     get
>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>>                                                     suite
>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>                                                     run
>>>>>>>>>>>>>>>>>>>                                                     manually
>>>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>>>                                                     Mellanox
>>>>>>>>>>>>>>>>>>>                                                     CX5
>>>>>>>>>>>>>>>>>>>                                                     devices
>>>>>>>>>>>>>>>>>>>                                                     once.
>>>>>>>>>>>>>>>>>>>                                                     However,
>>>>>>>>>>>>>>>>>>>                                                     we
>>>>>>>>>>>>>>>>>>>                                                     are
>>>>>>>>>>>>>>>>>>>                                                     running
>>>>>>>>>>>>>>>>>>>                                                     into
>>>>>>>>>>>>>>>>>>>                                                     an
>>>>>>>>>>>>>>>>>>>                                                     issue
>>>>>>>>>>>>>>>>>>>                                                     where,
>>>>>>>>>>>>>>>>>>>                                                     when
>>>>>>>>>>>>>>>>>>>                                                     RCF
>>>>>>>>>>>>>>>>>>>                                                     starts,
>>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>                                                     RCF
>>>>>>>>>>>>>>>>>>>                                                     connection
>>>>>>>>>>>>>>>>>>>                                                     times
>>>>>>>>>>>>>>>>>>>                                                     out
>>>>>>>>>>>>>>>>>>>                                                     very
>>>>>>>>>>>>>>>>>>>                                                     frequently.
>>>>>>>>>>>>>>>>>>>                                                     We
>>>>>>>>>>>>>>>>>>>                                                     aren't
>>>>>>>>>>>>>>>>>>>                                                     sure
>>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>>                                                     why
>>>>>>>>>>>>>>>>>>>                                                     this
>>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>                                                     case.
>>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>>                                                     It
>>>>>>>>>>>>>>>>>>>                                                     works
>>>>>>>>>>>>>>>>>>>                                                     sometimes,
>>>>>>>>>>>>>>>>>>>                                                     but
>>>>>>>>>>>>>>>>>>>                                                     most
>>>>>>>>>>>>>>>>>>>                                                     of
>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>                                                     time
>>>>>>>>>>>>>>>>>>>                                                     when
>>>>>>>>>>>>>>>>>>>                                                     we
>>>>>>>>>>>>>>>>>>>                                                     try
>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>                                                     run
>>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>>                                                     engine,
>>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>>                                                     encounters
>>>>>>>>>>>>>>>>>>>                                                     this
>>>>>>>>>>>>>>>>>>>                                                     issue.
>>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>>                                                     I've
>>>>>>>>>>>>>>>>>>>                                                     tried
>>>>>>>>>>>>>>>>>>>                                                     changing
>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>                                                     RCF
>>>>>>>>>>>>>>>>>>>                                                     port
>>>>>>>>>>>>>>>>>>>                                                     by
>>>>>>>>>>>>>>>>>>>                                                     setting
>>>>>>>>>>>>>>>>>>>                                                     "TE_RCF_PORT=<some
>>>>>>>>>>>>>>>>>>>                                                     port
>>>>>>>>>>>>>>>>>>>                                                     number>"
>>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>>                                                     rebooting
>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>                                                     testbed
>>>>>>>>>>>>>>>>>>>                                                     machines.
>>>>>>>>>>>>>>>>>>>                                                     Neither
>>>>>>>>>>>>>>>>>>>                                                     seems
>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>                                                     fix
>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>                                                     issue.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>>                                                     It
>>>>>>>>>>>>>>>>>>>                                                     also
>>>>>>>>>>>>>>>>>>>                                                     seems
>>>>>>>>>>>>>>>>>>>                                                     like
>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>                                                     timeout
>>>>>>>>>>>>>>>>>>>                                                     takes
>>>>>>>>>>>>>>>>>>>                                                     far
>>>>>>>>>>>>>>>>>>>                                                     longer
>>>>>>>>>>>>>>>>>>>                                                     than
>>>>>>>>>>>>>>>>>>>                                                     60
>>>>>>>>>>>>>>>>>>>                                                     seconds,
>>>>>>>>>>>>>>>>>>>                                                     even
>>>>>>>>>>>>>>>>>>>                                                     when
>>>>>>>>>>>>>>>>>>>                                                     running
>>>>>>>>>>>>>>>>>>>                                                     "export
>>>>>>>>>>>>>>>>>>>                                                     TE_RCFUNIX_TIMEOUT=60"
>>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>>                                                     before
>>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>>                                                     try
>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>                                                     run
>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>>                                                     suite.
>>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>>                                                     assume
>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>                                                     unit
>>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>>                                                     this
>>>>>>>>>>>>>>>>>>>                                                     variable
>>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>>                                                     seconds?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>                                                     Thanks,
>>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>>                                                     Adam
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>>                                                     On
>>>>>>>>>>>>>>>>>>>                                                     Mon,
>>>>>>>>>>>>>>>>>>>                                                     Aug
>>>>>>>>>>>>>>>>>>>                                                     21,
>>>>>>>>>>>>>>>>>>>                                                     2023
>>>>>>>>>>>>>>>>>>>                                                     at
>>>>>>>>>>>>>>>>>>>                                                     10:19 AM
>>>>>>>>>>>>>>>>>>>                                                     Adam
>>>>>>>>>>>>>>>>>>>                                                     Hassick
>>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>>                                                     <ahassick@iol.unh.edu
>>>>>>>>>>>>>>>>>>>                                                     <mailto:ahassick@iol.unh.edu>
>>>>>>>>>>>>>>>>>>>                                                     <mailto:ahassick@iol.unh.edu>>
>>>>>>>>>>>>>>>>>>>                                                     wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>                                                                
>>>>>>>>>>>>>>>>>>>                                                     Hi
>>>>>>>>>>>>>>>>>>>                                                     Andrew,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>                                                     Thanks,
>>>>>>>>>>>>>>>>>>>                                                     I've
>>>>>>>>>>>>>>>>>>>                                                     cloned
>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>                                                     example
>>>>>>>>>>>>>>>>>>>                                                     repository
>>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>>                                                     will
>>>>>>>>>>>>>>>>>>>                                                     start
>>>>>>>>>>>>>>>>>>>                                                     setting
>>>>>>>>>>>>>>>>>>>                                                     up
>>>>>>>>>>>>>>>>>>>                                                     a
>>>>>>>>>>>>>>>>>>>                                                     configuration
>>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>>                                                     our
>>>>>>>>>>>>>>>>>>>                                                     development
>>>>>>>>>>>>>>>>>>>                                                     testbed
>>>>>>>>>>>>>>>>>>>                                                     today.
>>>>>>>>>>>>>>>>>>>                                                     I'll
>>>>>>>>>>>>>>>>>>>                                                     let
>>>>>>>>>>>>>>>>>>>                                                     you
>>>>>>>>>>>>>>>>>>>                                                     know
>>>>>>>>>>>>>>>>>>>                                                     if
>>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>>                                                     run
>>>>>>>>>>>>>>>>>>>                                                     into
>>>>>>>>>>>>>>>>>>>                                                     any
>>>>>>>>>>>>>>>>>>>                                                     difficulties
>>>>>>>>>>>>>>>>>>>                                                                
>>>>>>>>>>>>>>>>>>>                                                     or
>>>>>>>>>>>>>>>>>>>                                                     have
>>>>>>>>>>>>>>>>>>>                                                     any
>>>>>>>>>>>>>>>>>>>                                                     questions.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>                                                                
>>>>>>>>>>>>>>>>>>>                                                      -
>>>>>>>>>>>>>>>>>>>                                                     Adam
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>                                                                
>>>>>>>>>>>>>>>>>>>                                                     On
>>>>>>>>>>>>>>>>>>>                                                     Sun,
>>>>>>>>>>>>>>>>>>>                                                     Aug
>>>>>>>>>>>>>>>>>>>                                                     20,
>>>>>>>>>>>>>>>>>>>                                                     2023
>>>>>>>>>>>>>>>>>>>                                                     at
>>>>>>>>>>>>>>>>>>>                                                     4:40 AM
>>>>>>>>>>>>>>>>>>>                                                     Andrew
>>>>>>>>>>>>>>>>>>>                                                     Rybchenko
>>>>>>>>>>>>>>>>>>>                                                     <andrew.rybchenko@oktetlabs.ru
>>>>>>>>>>>>>>>>>>>                                                     <mailto:andrew.rybchenko@oktetlabs.ru>
>>>>>>>>>>>>>>>>>>>                                                     <mailto:andrew.rybchenko@oktetlabs.ru>>
>>>>>>>>>>>>>>>>>>>                                                     wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>                                                     Hi
>>>>>>>>>>>>>>>>>>>                                                     Adam,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>                                                     I've
>>>>>>>>>>>>>>>>>>>                                                     published
>>>>>>>>>>>>>>>>>>>                                                     https://github.com/ts-factory/ts-rigs-sample
>>>>>>>>>>>>>>>>>>>                                                     <https://github.com/ts-factory/ts-rigs-sample>
>>>>>>>>>>>>>>>>>>>                                                     <https://github.com/ts-factory/ts-rigs-sample>.
>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>                                                     Hopefully
>>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>>                                                     will
>>>>>>>>>>>>>>>>>>>                                                     help
>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>                                                     define
>>>>>>>>>>>>>>>>>>>                                                     your
>>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>>                                                     rigs
>>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>                                                     successfully
>>>>>>>>>>>>>>>>>>>                                                     run
>>>>>>>>>>>>>>>>>>>                                                     some
>>>>>>>>>>>>>>>>>>>                                                     tests
>>>>>>>>>>>>>>>>>>>                                                     manually.
>>>>>>>>>>>>>>>>>>>                                                     Feel
>>>>>>>>>>>>>>>>>>>                                                     free
>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>                                                     ask
>>>>>>>>>>>>>>>>>>>                                                     any
>>>>>>>>>>>>>>>>>>>                                                     questions
>>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>>                                                     I'll
>>>>>>>>>>>>>>>>>>>                                                     answer
>>>>>>>>>>>>>>>>>>>                                                     here
>>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>>                                                     try
>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>                                                     update
>>>>>>>>>>>>>>>>>>>                                                     documentation.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>                                                     Meanwhile
>>>>>>>>>>>>>>>>>>>                                                     I'll
>>>>>>>>>>>>>>>>>>>                                                     prepare
>>>>>>>>>>>>>>>>>>>                                                     missing
>>>>>>>>>>>>>>>>>>>                                                     bits
>>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>>                                                     steps
>>>>>>>>>>>>>>>>>>>                                                     (2)
>>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>                                                     (3).
>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>                                                     Hopefully
>>>>>>>>>>>>>>>>>>>                                                     everything
>>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>>                                                     place
>>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>>                                                     step
>>>>>>>>>>>>>>>>>>>                                                     (4),
>>>>>>>>>>>>>>>>>>>                                                     but
>>>>>>>>>>>>>>>>>>>                                                     we
>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>                                                     need
>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>                                                     make
>>>>>>>>>>>>>>>>>>>                                                     steps
>>>>>>>>>>>>>>>>>>>                                                     (2)
>>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>>                                                     (3)
>>>>>>>>>>>>>>>>>>>                                                     first.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>                                                     Andrew.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>                                                     On
>>>>>>>>>>>>>>>>>>>                                                     8/18/23
>>>>>>>>>>>>>>>>>>>                                                     21:40,
>>>>>>>>>>>>>>>>>>>                                                     Andrew
>>>>>>>>>>>>>>>>>>>                                                     Rybchenko
>>>>>>>>>>>>>>>>>>>                                                     wrote:
>>>>>>>>>>>>>>>>>>>>                                                     Hi
>>>>>>>>>>>>>>>>>>>>                                                     Adam,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>                                                     >
>>>>>>>>>>>>>>>>>>>>                                                     I've
>>>>>>>>>>>>>>>>>>>>                                                     conferred
>>>>>>>>>>>>>>>>>>>>                                                     with
>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>                                                     rest
>>>>>>>>>>>>>>>>>>>>                                                     of
>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>                                                     team,
>>>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>>>                                                     we
>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>                                                     think
>>>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>>>                                                     would
>>>>>>>>>>>>>>>>>>>>                                                     be
>>>>>>>>>>>>>>>>>>>>                                                     best
>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>                                                     move
>>>>>>>>>>>>>>>>>>>>                                                     forward
>>>>>>>>>>>>>>>>>>>>                                                     with
>>>>>>>>>>>>>>>>>>>>                                                     mainly
>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>                                                     option
>>>>>>>>>>>>>>>>>>>>                                                     B.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>                                                     OK,
>>>>>>>>>>>>>>>>>>>>                                                     I'll
>>>>>>>>>>>>>>>>>>>>                                                     provide
>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>                                                     sample
>>>>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>>>>                                                     Monday
>>>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>>>                                                     you.
>>>>>>>>>>>>>>>>>>>>                                                     It
>>>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>                                                     almost
>>>>>>>>>>>>>>>>>>>>                                                     ready
>>>>>>>>>>>>>>>>>>>>                                                     right
>>>>>>>>>>>>>>>>>>>>                                                     now,
>>>>>>>>>>>>>>>>>>>>                                                     but
>>>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>>>                                                     need
>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>                                                     double-check
>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>>>                                                     before
>>>>>>>>>>>>>>>>>>>>                                                     publishing.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>                                                     Regards,
>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>                                                     Andrew.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>                                                     On
>>>>>>>>>>>>>>>>>>>>                                                     8/17/23
>>>>>>>>>>>>>>>>>>>>                                                     20:03,
>>>>>>>>>>>>>>>>>>>>                                                     Adam
>>>>>>>>>>>>>>>>>>>>                                                     Hassick
>>>>>>>>>>>>>>>>>>>>                                                     wrote:
>>>>>>>>>>>>>>>>>>>>>                                                     Hi
>>>>>>>>>>>>>>>>>>>>>                                                     Andrew,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     I'm
>>>>>>>>>>>>>>>>>>>>>                                                     adding
>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>                                                     CI
>>>>>>>>>>>>>>>>>>>>>                                                     mailing
>>>>>>>>>>>>>>>>>>>>>                                                     list
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     this
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     conversation.
>>>>>>>>>>>>>>>>>>>>>                                                     Others
>>>>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>                                                     community
>>>>>>>>>>>>>>>>>>>>>                                                     might
>>>>>>>>>>>>>>>>>>>>>                                                     find
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     this
>>>>>>>>>>>>>>>>>>>>>                                                     conversation
>>>>>>>>>>>>>>>>>>>>>                                                     valuable.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     We
>>>>>>>>>>>>>>>>>>>>>                                                     do
>>>>>>>>>>>>>>>>>>>>>                                                     want
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     run
>>>>>>>>>>>>>>>>>>>>>                                                     testing
>>>>>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>>>>>                                                     a
>>>>>>>>>>>>>>>>>>>>>                                                     regular
>>>>>>>>>>>>>>>>>>>>>                                                     basis.
>>>>>>>>>>>>>>>>>>>>>                                                     The
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     Jenkins
>>>>>>>>>>>>>>>>>>>>>                                                     integration
>>>>>>>>>>>>>>>>>>>>>                                                     will
>>>>>>>>>>>>>>>>>>>>>                                                     be
>>>>>>>>>>>>>>>>>>>>>                                                     very
>>>>>>>>>>>>>>>>>>>>>                                                     useful
>>>>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>>>>                                                     us,
>>>>>>>>>>>>>>>>>>>>>                                                     as
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     most
>>>>>>>>>>>>>>>>>>>>>                                                     of
>>>>>>>>>>>>>>>>>>>>>                                                     our
>>>>>>>>>>>>>>>>>>>>>                                                     CI
>>>>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>>>>                                                     orchestrated
>>>>>>>>>>>>>>>>>>>>>                                                     by
>>>>>>>>>>>>>>>>>>>>>                                                     Jenkins.
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     I've
>>>>>>>>>>>>>>>>>>>>>                                                     conferred
>>>>>>>>>>>>>>>>>>>>>                                                     with
>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>                                                     rest
>>>>>>>>>>>>>>>>>>>>>                                                     of
>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>                                                     team,
>>>>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>>>>                                                     we
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     think
>>>>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>>>>                                                     would
>>>>>>>>>>>>>>>>>>>>>                                                     be
>>>>>>>>>>>>>>>>>>>>>                                                     best
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     move
>>>>>>>>>>>>>>>>>>>>>                                                     forward
>>>>>>>>>>>>>>>>>>>>>                                                     with
>>>>>>>>>>>>>>>>>>>>>                                                     mainly
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     option
>>>>>>>>>>>>>>>>>>>>>                                                     B.
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     If
>>>>>>>>>>>>>>>>>>>>>                                                     you
>>>>>>>>>>>>>>>>>>>>>                                                     would
>>>>>>>>>>>>>>>>>>>>>                                                     like
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     know
>>>>>>>>>>>>>>>>>>>>>                                                     anything
>>>>>>>>>>>>>>>>>>>>>                                                     about
>>>>>>>>>>>>>>>>>>>>>                                                     our
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     testbeds
>>>>>>>>>>>>>>>>>>>>>                                                     that
>>>>>>>>>>>>>>>>>>>>>                                                     would
>>>>>>>>>>>>>>>>>>>>>                                                     help
>>>>>>>>>>>>>>>>>>>>>                                                     you
>>>>>>>>>>>>>>>>>>>>>                                                     with
>>>>>>>>>>>>>>>>>>>>>                                                     creating
>>>>>>>>>>>>>>>>>>>>>                                                     an
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     example
>>>>>>>>>>>>>>>>>>>>>                                                     ts-rigs
>>>>>>>>>>>>>>>>>>>>>                                                     repo,
>>>>>>>>>>>>>>>>>>>>>                                                     I'd
>>>>>>>>>>>>>>>>>>>>>                                                     be
>>>>>>>>>>>>>>>>>>>>>                                                     happy
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     answer
>>>>>>>>>>>>>>>>>>>>>                                                     any
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     questions
>>>>>>>>>>>>>>>>>>>>>                                                     you
>>>>>>>>>>>>>>>>>>>>>                                                     have.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     We
>>>>>>>>>>>>>>>>>>>>>                                                     have
>>>>>>>>>>>>>>>>>>>>>                                                     multiple
>>>>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>>>>                                                     rigs
>>>>>>>>>>>>>>>>>>>>>                                                     (we
>>>>>>>>>>>>>>>>>>>>>                                                     call
>>>>>>>>>>>>>>>>>>>>>                                                     these
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     "DUT-tester
>>>>>>>>>>>>>>>>>>>>>                                                     pairs")
>>>>>>>>>>>>>>>>>>>>>                                                     that
>>>>>>>>>>>>>>>>>>>>>                                                     we
>>>>>>>>>>>>>>>>>>>>>                                                     run
>>>>>>>>>>>>>>>>>>>>>                                                     our
>>>>>>>>>>>>>>>>>>>>>                                                     existing
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     hardware
>>>>>>>>>>>>>>>>>>>>>                                                     testing
>>>>>>>>>>>>>>>>>>>>>                                                     on,
>>>>>>>>>>>>>>>>>>>>>                                                     with
>>>>>>>>>>>>>>>>>>>>>                                                     differing
>>>>>>>>>>>>>>>>>>>>>                                                     network
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     hardware
>>>>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>>>>                                                     CPU
>>>>>>>>>>>>>>>>>>>>>                                                     architecture.
>>>>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>>>>                                                     figured
>>>>>>>>>>>>>>>>>>>>>                                                     this
>>>>>>>>>>>>>>>>>>>>>                                                     might
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     be
>>>>>>>>>>>>>>>>>>>>>                                                     an
>>>>>>>>>>>>>>>>>>>>>                                                     important
>>>>>>>>>>>>>>>>>>>>>                                                     detail.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     Thanks,
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     Adam
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     On
>>>>>>>>>>>>>>>>>>>>>                                                     Thu,
>>>>>>>>>>>>>>>>>>>>>                                                     Aug
>>>>>>>>>>>>>>>>>>>>>                                                     17,
>>>>>>>>>>>>>>>>>>>>>                                                     2023
>>>>>>>>>>>>>>>>>>>>>                                                     at
>>>>>>>>>>>>>>>>>>>>>                                                     11:44 AM
>>>>>>>>>>>>>>>>>>>>>                                                     Andrew
>>>>>>>>>>>>>>>>>>>>>                                                     Rybchenko
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     <andrew.rybchenko@oktetlabs.ru
>>>>>>>>>>>>>>>>>>>>>                                                     <mailto:andrew.rybchenko@oktetlabs.ru>
>>>>>>>>>>>>>>>>>>>>>                                                     <mailto:andrew.rybchenko@oktetlabs.ru>>
>>>>>>>>>>>>>>>>>>>>>                                                     wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     Greatings
>>>>>>>>>>>>>>>>>>>>>                                                     Adam,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     I'm
>>>>>>>>>>>>>>>>>>>>>                                                     happy
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     hear
>>>>>>>>>>>>>>>>>>>>>                                                     that
>>>>>>>>>>>>>>>>>>>>>                                                     you're
>>>>>>>>>>>>>>>>>>>>>                                                     trying
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     bring
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>>>>                                                     up.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     As
>>>>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>>>>                                                     understand
>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>                                                     final
>>>>>>>>>>>>>>>>>>>>>                                                     goal
>>>>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     run
>>>>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     regular
>>>>>>>>>>>>>>>>>>>>>                                                     basis.
>>>>>>>>>>>>>>>>>>>>>                                                     So,
>>>>>>>>>>>>>>>>>>>>>                                                     we
>>>>>>>>>>>>>>>>>>>>>                                                     need
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     make
>>>>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>>>>                                                     properly
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     from
>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>                                                     very
>>>>>>>>>>>>>>>>>>>>>                                                     beginning.
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     Bring
>>>>>>>>>>>>>>>>>>>>>                                                     up
>>>>>>>>>>>>>>>>>>>>>                                                     of
>>>>>>>>>>>>>>>>>>>>>                                                     all
>>>>>>>>>>>>>>>>>>>>>                                                     features
>>>>>>>>>>>>>>>>>>>>>                                                     consists
>>>>>>>>>>>>>>>>>>>>>                                                     of
>>>>>>>>>>>>>>>>>>>>>                                                     4
>>>>>>>>>>>>>>>>>>>>>                                                     steps:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     1.
>>>>>>>>>>>>>>>>>>>>>                                                     Create
>>>>>>>>>>>>>>>>>>>>>                                                     site-specific
>>>>>>>>>>>>>>>>>>>>>                                                     repository
>>>>>>>>>>>>>>>>>>>>>                                                     (we
>>>>>>>>>>>>>>>>>>>>>                                                     call
>>>>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     ts-rigs)
>>>>>>>>>>>>>>>>>>>>>                                                     which
>>>>>>>>>>>>>>>>>>>>>                                                     contains
>>>>>>>>>>>>>>>>>>>>>                                                     information
>>>>>>>>>>>>>>>>>>>>>                                                     about
>>>>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     rigs
>>>>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>>>>                                                     other
>>>>>>>>>>>>>>>>>>>>>                                                     site-specific
>>>>>>>>>>>>>>>>>>>>>                                                     information
>>>>>>>>>>>>>>>>>>>>>                                                     like
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     where
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     send
>>>>>>>>>>>>>>>>>>>>>                                                     mails,
>>>>>>>>>>>>>>>>>>>>>                                                     where
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     store
>>>>>>>>>>>>>>>>>>>>>                                                     logs
>>>>>>>>>>>>>>>>>>>>>                                                     etc.
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     It
>>>>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>>>>                                                     required
>>>>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>>>>                                                     manual
>>>>>>>>>>>>>>>>>>>>>                                                     execution
>>>>>>>>>>>>>>>>>>>>>                                                     as
>>>>>>>>>>>>>>>>>>>>>                                                     well,
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     since
>>>>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>>>>                                                     rigs
>>>>>>>>>>>>>>>>>>>>>                                                     description
>>>>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>>>>                                                     essential.
>>>>>>>>>>>>>>>>>>>>>                                                     I'll
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     return
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>                                                     topic
>>>>>>>>>>>>>>>>>>>>>                                                     below.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     2.
>>>>>>>>>>>>>>>>>>>>>                                                     Setup
>>>>>>>>>>>>>>>>>>>>>                                                     logs
>>>>>>>>>>>>>>>>>>>>>                                                     storage
>>>>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>>>>                                                     automated
>>>>>>>>>>>>>>>>>>>>>                                                     runs.
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     Basically
>>>>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>>>>                                                     a
>>>>>>>>>>>>>>>>>>>>>                                                     disk
>>>>>>>>>>>>>>>>>>>>>                                                     space
>>>>>>>>>>>>>>>>>>>>>                                                     plus
>>>>>>>>>>>>>>>>>>>>>                                                     apache2
>>>>>>>>>>>>>>>>>>>>>                                                     web
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     server
>>>>>>>>>>>>>>>>>>>>>                                                     with
>>>>>>>>>>>>>>>>>>>>>                                                     few
>>>>>>>>>>>>>>>>>>>>>                                                     CGI
>>>>>>>>>>>>>>>>>>>>>                                                     scripts
>>>>>>>>>>>>>>>>>>>>>                                                     which
>>>>>>>>>>>>>>>>>>>>>                                                     help
>>>>>>>>>>>>>>>>>>>>>                                                     a
>>>>>>>>>>>>>>>>>>>>>                                                     lot
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     save
>>>>>>>>>>>>>>>>>>>>>                                                     disk
>>>>>>>>>>>>>>>>>>>>>                                                     space.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     3.
>>>>>>>>>>>>>>>>>>>>>                                                     Setup
>>>>>>>>>>>>>>>>>>>>>                                                     Bublik
>>>>>>>>>>>>>>>>>>>>>                                                     web
>>>>>>>>>>>>>>>>>>>>>                                                     application
>>>>>>>>>>>>>>>>>>>>>                                                     which
>>>>>>>>>>>>>>>>>>>>>                                                     provides
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     web
>>>>>>>>>>>>>>>>>>>>>                                                     interface
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     view
>>>>>>>>>>>>>>>>>>>>>                                                     testing
>>>>>>>>>>>>>>>>>>>>>                                                     results.
>>>>>>>>>>>>>>>>>>>>>                                                     Same
>>>>>>>>>>>>>>>>>>>>>                                                     as
>>>>>>>>>>>>>>>>>>>>>                                                     https://ts-factory.io/bublik
>>>>>>>>>>>>>>>>>>>>>                                                     <https://ts-factory.io/bublik>
>>>>>>>>>>>>>>>>>>>>>                                                     <https://ts-factory.io/bublik>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     4.
>>>>>>>>>>>>>>>>>>>>>                                                     Setup
>>>>>>>>>>>>>>>>>>>>>                                                     Jenkins
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     run
>>>>>>>>>>>>>>>>>>>>>                                                     tests
>>>>>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>>>>>                                                     regularly,
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     save
>>>>>>>>>>>>>>>>>>>>>                                                     logs
>>>>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>>>>                                                     log
>>>>>>>>>>>>>>>>>>>>>                                                     storage
>>>>>>>>>>>>>>>>>>>>>                                                     (2)
>>>>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>>>>                                                     import
>>>>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     bublik
>>>>>>>>>>>>>>>>>>>>>                                                     (3).
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     Last
>>>>>>>>>>>>>>>>>>>>>                                                     few
>>>>>>>>>>>>>>>>>>>>>                                                     month
>>>>>>>>>>>>>>>>>>>>>                                                     we
>>>>>>>>>>>>>>>>>>>>>                                                     spent
>>>>>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>>>>>                                                     our
>>>>>>>>>>>>>>>>>>>>>                                                     homework
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     make
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>>>>                                                     simpler
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     bring
>>>>>>>>>>>>>>>>>>>>>                                                     up
>>>>>>>>>>>>>>>>>>>>>                                                     automated
>>>>>>>>>>>>>>>>>>>>>                                                     execution
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     using
>>>>>>>>>>>>>>>>>>>>>                                                     Jenkins
>>>>>>>>>>>>>>>>>>>>>                                                     -
>>>>>>>>>>>>>>>>>>>>>                                                     https://github.com/ts-factory/te-jenkins
>>>>>>>>>>>>>>>>>>>>>                                                     <https://github.com/ts-factory/te-jenkins>
>>>>>>>>>>>>>>>>>>>>>                                                     <https://github.com/ts-factory/te-jenkins>
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     Corresponding
>>>>>>>>>>>>>>>>>>>>>                                                     bits
>>>>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>>>>                                                     dpdk-ethdev-ts
>>>>>>>>>>>>>>>>>>>>>                                                     will
>>>>>>>>>>>>>>>>>>>>>                                                     be
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     available
>>>>>>>>>>>>>>>>>>>>>                                                     tomorrow.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     Let's
>>>>>>>>>>>>>>>>>>>>>                                                     return
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>                                                     step
>>>>>>>>>>>>>>>>>>>>>                                                     (1).
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     Unfortunately
>>>>>>>>>>>>>>>>>>>>>                                                     there
>>>>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>>>>                                                     no
>>>>>>>>>>>>>>>>>>>>>                                                     publicly
>>>>>>>>>>>>>>>>>>>>>                                                     available
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     example
>>>>>>>>>>>>>>>>>>>>>                                                     of
>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>                                                     ts-rigs
>>>>>>>>>>>>>>>>>>>>>                                                     repository
>>>>>>>>>>>>>>>>>>>>>                                                     since
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     sensitive
>>>>>>>>>>>>>>>>>>>>>                                                     site-specific
>>>>>>>>>>>>>>>>>>>>>                                                     information
>>>>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>>>>                                                     located
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     there.
>>>>>>>>>>>>>>>>>>>>>                                                     But
>>>>>>>>>>>>>>>>>>>>>                                                     I'm
>>>>>>>>>>>>>>>>>>>>>                                                     ready
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     help
>>>>>>>>>>>>>>>>>>>>>                                                     you
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     create
>>>>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>>>>                                                     UNH.
>>>>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>>>>                                                     see
>>>>>>>>>>>>>>>>>>>>>                                                     two
>>>>>>>>>>>>>>>>>>>>>                                                     options
>>>>>>>>>>>>>>>>>>>>>                                                     here:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     (A)
>>>>>>>>>>>>>>>>>>>>>                                                     I'll
>>>>>>>>>>>>>>>>>>>>>                                                     ask
>>>>>>>>>>>>>>>>>>>>>                                                     questions
>>>>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>>>>                                                     based
>>>>>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>>>>>                                                     your
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     answers
>>>>>>>>>>>>>>>>>>>>>                                                     will
>>>>>>>>>>>>>>>>>>>>>                                                     create
>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>                                                     first
>>>>>>>>>>>>>>>>>>>>>                                                     draft
>>>>>>>>>>>>>>>>>>>>>                                                     with
>>>>>>>>>>>>>>>>>>>>>                                                     my
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     comments.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     (B)
>>>>>>>>>>>>>>>>>>>>>                                                     I'll
>>>>>>>>>>>>>>>>>>>>>                                                     make
>>>>>>>>>>>>>>>>>>>>>                                                     a
>>>>>>>>>>>>>>>>>>>>>                                                     template/example
>>>>>>>>>>>>>>>>>>>>>                                                     ts-rigs
>>>>>>>>>>>>>>>>>>>>>                                                     repo,
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     publish
>>>>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>>>>                                                     you'll
>>>>>>>>>>>>>>>>>>>>>                                                     create
>>>>>>>>>>>>>>>>>>>>>                                                     UNH
>>>>>>>>>>>>>>>>>>>>>                                                     ts-rigs
>>>>>>>>>>>>>>>>>>>>>                                                     based
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>>>>>                                                     it.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     Of
>>>>>>>>>>>>>>>>>>>>>                                                     course,
>>>>>>>>>>>>>>>>>>>>>                                                     I'll
>>>>>>>>>>>>>>>>>>>>>                                                     help
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     debug
>>>>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>>>>                                                     finally
>>>>>>>>>>>>>>>>>>>>>                                                     bring
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>>>>                                                     up
>>>>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>>>>                                                     any
>>>>>>>>>>>>>>>>>>>>>                                                     case.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     (A)
>>>>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>>>>                                                     a
>>>>>>>>>>>>>>>>>>>>>                                                     bit
>>>>>>>>>>>>>>>>>>>>>                                                     simpler
>>>>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>>>>                                                     me
>>>>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>>>>                                                     you,
>>>>>>>>>>>>>>>>>>>>>                                                     but
>>>>>>>>>>>>>>>>>>>>>                                                     (B)
>>>>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     a
>>>>>>>>>>>>>>>>>>>>>                                                     bit
>>>>>>>>>>>>>>>>>>>>>                                                     more
>>>>>>>>>>>>>>>>>>>>>                                                     generic
>>>>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>>>>                                                     will
>>>>>>>>>>>>>>>>>>>>>                                                     help
>>>>>>>>>>>>>>>>>>>>>                                                     other
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     potential
>>>>>>>>>>>>>>>>>>>>>                                                     users
>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>                                                     bring
>>>>>>>>>>>>>>>>>>>>>                                                     it
>>>>>>>>>>>>>>>>>>>>>                                                     up.
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     We
>>>>>>>>>>>>>>>>>>>>>                                                     can
>>>>>>>>>>>>>>>>>>>>>                                                     combine
>>>>>>>>>>>>>>>>>>>>>                                                     (A)+(B).
>>>>>>>>>>>>>>>>>>>>>                                                     I.e.
>>>>>>>>>>>>>>>>>>>>>                                                     start
>>>>>>>>>>>>>>>>>>>>>                                                     from
>>>>>>>>>>>>>>>>>>>>>                                                     (A).
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     What
>>>>>>>>>>>>>>>>>>>>>                                                     do
>>>>>>>>>>>>>>>>>>>>>                                                     you
>>>>>>>>>>>>>>>>>>>>>                                                     think?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     Thanks,
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     Andrew.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>                                                     On
>>>>>>>>>>>>>>>>>>>>>                                                     8/17/23
>>>>>>>>>>>>>>>>>>>>>                                                     15:18,
>>>>>>>>>>>>>>>>>>>>>                                                     Konstantin
>>>>>>>>>>>>>>>>>>>>>                                                     Ushakov
>>>>>>>>>>>>>>>>>>>>>                                                     wrote:
>>>>>>>>>>>>>>>>>>>>>>                                                     Greetings
>>>>>>>>>>>>>>>>>>>>>>                                                     Adam,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>                                                     Thanks
>>>>>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>>>>>                                                     contacting
>>>>>>>>>>>>>>>>>>>>>>                                                     us.
>>>>>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>>>>>                                                     copy
>>>>>>>>>>>>>>>>>>>>>>                                                     Andrew
>>>>>>>>>>>>>>>>>>>>>>                                                     who
>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>                                                     would
>>>>>>>>>>>>>>>>>>>>>>                                                     be
>>>>>>>>>>>>>>>>>>>>>>                                                     happy
>>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>>                                                     help
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>                                                     Thanks,
>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>                                                     Konstantin
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>                                                     On
>>>>>>>>>>>>>>>>>>>>>>>                                                     16
>>>>>>>>>>>>>>>>>>>>>>>                                                     Aug
>>>>>>>>>>>>>>>>>>>>>>>                                                     2023,
>>>>>>>>>>>>>>>>>>>>>>>                                                     at
>>>>>>>>>>>>>>>>>>>>>>>                                                     21:50,
>>>>>>>>>>>>>>>>>>>>>>>                                                     Adam
>>>>>>>>>>>>>>>>>>>>>>>                                                     Hassick
>>>>>>>>>>>>>>>>>>>>>>>                                                     <ahassick@iol.unh.edu>
>>>>>>>>>>>>>>>>>>>>>>>                                                     <mailto:ahassick@iol.unh.edu>
>>>>>>>>>>>>>>>>>>>>>>>                                                     <mailto:ahassick@iol.unh.edu>
>>>>>>>>>>>>>>>>>>>>>>>                                                     <mailto:ahassick@iol.unh.edu>
>>>>>>>>>>>>>>>>>>>>>>>                                                     wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     Greetings
>>>>>>>>>>>>>>>>>>>>>>>                                                     Konstantin,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>>>>>>                                                     am
>>>>>>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>>>                                                     process
>>>>>>>>>>>>>>>>>>>>>>>                                                     of
>>>>>>>>>>>>>>>>>>>>>>>                                                     setting
>>>>>>>>>>>>>>>>>>>>>>>                                                     up
>>>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>>>                                                     DPDK
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     Poll
>>>>>>>>>>>>>>>>>>>>>>>                                                     Mode
>>>>>>>>>>>>>>>>>>>>>>>                                                     Driver
>>>>>>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>>>>>>                                                     suite
>>>>>>>>>>>>>>>>>>>>>>>                                                     as
>>>>>>>>>>>>>>>>>>>>>>>                                                     an
>>>>>>>>>>>>>>>>>>>>>>>                                                     addition
>>>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     our
>>>>>>>>>>>>>>>>>>>>>>>                                                     testing
>>>>>>>>>>>>>>>>>>>>>>>                                                     coverage
>>>>>>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>>>>>>                                                     DPDK
>>>>>>>>>>>>>>>>>>>>>>>                                                     at
>>>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>>>                                                     UNH
>>>>>>>>>>>>>>>>>>>>>>>                                                     lab.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>>>>>>                                                     have
>>>>>>>>>>>>>>>>>>>>>>>                                                     some
>>>>>>>>>>>>>>>>>>>>>>>                                                     questions
>>>>>>>>>>>>>>>>>>>>>>>                                                     about
>>>>>>>>>>>>>>>>>>>>>>>                                                     how
>>>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>>>                                                     set
>>>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>>>>>>                                                     suite
>>>>>>>>>>>>>>>>>>>>>>>                                                     arguments.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>>>>>>                                                     have
>>>>>>>>>>>>>>>>>>>>>>>                                                     been
>>>>>>>>>>>>>>>>>>>>>>>                                                     able
>>>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>>>                                                     configure
>>>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>>>                                                     Test
>>>>>>>>>>>>>>>>>>>>>>>                                                     Engine
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>>>                                                     connect
>>>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>>>                                                     hosts
>>>>>>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>>>                                                     testbed.
>>>>>>>>>>>>>>>>>>>>>>>                                                     The
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     RCF,
>>>>>>>>>>>>>>>>>>>>>>>                                                     Configurator,
>>>>>>>>>>>>>>>>>>>>>>>                                                     and
>>>>>>>>>>>>>>>>>>>>>>>                                                     Tester
>>>>>>>>>>>>>>>>>>>>>>>                                                     all
>>>>>>>>>>>>>>>>>>>>>>>                                                     begin
>>>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     run,
>>>>>>>>>>>>>>>>>>>>>>>                                                     however
>>>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>>>                                                     prelude
>>>>>>>>>>>>>>>>>>>>>>>                                                     of
>>>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>>>>>>                                                     suite
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     fails
>>>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>>>                                                     run.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>                                                     https://ts-factory.io/doc/dpdk-ethdev-ts/index.html#test-parameters
>>>>>>>>>>>>>>>>>>>>>>>                                                     <https://ts-factory.io/doc/dpdk-ethdev-ts/index.html#test-parameters>
>>>>>>>>>>>>>>>>>>>>>>>                                                     <https://ts-factory.io/doc/dpdk-ethdev-ts/index.html#test-parameters>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     The
>>>>>>>>>>>>>>>>>>>>>>>                                                     documentation
>>>>>>>>>>>>>>>>>>>>>>>                                                     mentions
>>>>>>>>>>>>>>>>>>>>>>>                                                     that
>>>>>>>>>>>>>>>>>>>>>>>                                                     there
>>>>>>>>>>>>>>>>>>>>>>>                                                     are
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     several
>>>>>>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>>>>>>                                                     parameters
>>>>>>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>>>>>>                                                     suite,
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     like
>>>>>>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>>>                                                     IUT
>>>>>>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>>>>>>                                                     link
>>>>>>>>>>>>>>>>>>>>>>>                                                     MAC,
>>>>>>>>>>>>>>>>>>>>>>>                                                     etc.
>>>>>>>>>>>>>>>>>>>>>>>                                                     These
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     seem
>>>>>>>>>>>>>>>>>>>>>>>                                                     like
>>>>>>>>>>>>>>>>>>>>>>>                                                     they
>>>>>>>>>>>>>>>>>>>>>>>                                                     would
>>>>>>>>>>>>>>>>>>>>>>>                                                     need
>>>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>>>                                                     be
>>>>>>>>>>>>>>>>>>>>>>>                                                     set
>>>>>>>>>>>>>>>>>>>>>>>                                                     somewhere
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>>>                                                     run
>>>>>>>>>>>>>>>>>>>>>>>                                                     many
>>>>>>>>>>>>>>>>>>>>>>>                                                     of
>>>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>>>                                                     tests.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>>>>>>                                                     see
>>>>>>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>>>                                                     Test
>>>>>>>>>>>>>>>>>>>>>>>                                                     Engine
>>>>>>>>>>>>>>>>>>>>>>>                                                     documentation,
>>>>>>>>>>>>>>>>>>>>>>>                                                     there
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     are
>>>>>>>>>>>>>>>>>>>>>>>                                                     instructions
>>>>>>>>>>>>>>>>>>>>>>>                                                     on
>>>>>>>>>>>>>>>>>>>>>>>                                                     how
>>>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>>>                                                     create
>>>>>>>>>>>>>>>>>>>>>>>                                                     new
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     parameters
>>>>>>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>>>>>>                                                     suites
>>>>>>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>>>                                                     Tester
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     configuration,
>>>>>>>>>>>>>>>>>>>>>>>                                                     but
>>>>>>>>>>>>>>>>>>>>>>>                                                     there
>>>>>>>>>>>>>>>>>>>>>>>                                                     is
>>>>>>>>>>>>>>>>>>>>>>>                                                     nothing
>>>>>>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     user
>>>>>>>>>>>>>>>>>>>>>>>                                                     guide
>>>>>>>>>>>>>>>>>>>>>>>                                                     or
>>>>>>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>>>                                                     Tester
>>>>>>>>>>>>>>>>>>>>>>>                                                     guide
>>>>>>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>>>>>>                                                     how
>>>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     set
>>>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>>>                                                     arguments
>>>>>>>>>>>>>>>>>>>>>>>                                                     for
>>>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>>>                                                     parameters
>>>>>>>>>>>>>>>>>>>>>>>                                                     when
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     running
>>>>>>>>>>>>>>>>>>>>>>>                                                     the
>>>>>>>>>>>>>>>>>>>>>>>                                                     test
>>>>>>>>>>>>>>>>>>>>>>>                                                     suite
>>>>>>>>>>>>>>>>>>>>>>>                                                     that
>>>>>>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>>>>>>                                                     can
>>>>>>>>>>>>>>>>>>>>>>>                                                     find.
>>>>>>>>>>>>>>>>>>>>>>>                                                     I'm
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     not
>>>>>>>>>>>>>>>>>>>>>>>                                                     sure
>>>>>>>>>>>>>>>>>>>>>>>                                                     if
>>>>>>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>>>>>>                                                     need
>>>>>>>>>>>>>>>>>>>>>>>                                                     to
>>>>>>>>>>>>>>>>>>>>>>>                                                     write
>>>>>>>>>>>>>>>>>>>>>>>                                                     my
>>>>>>>>>>>>>>>>>>>>>>>                                                     own
>>>>>>>>>>>>>>>>>>>>>>>                                                     Tester
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     config,
>>>>>>>>>>>>>>>>>>>>>>>                                                     or
>>>>>>>>>>>>>>>>>>>>>>>                                                     if
>>>>>>>>>>>>>>>>>>>>>>>                                                     I
>>>>>>>>>>>>>>>>>>>>>>>                                                     should
>>>>>>>>>>>>>>>>>>>>>>>                                                     be
>>>>>>>>>>>>>>>>>>>>>>>                                                     setting
>>>>>>>>>>>>>>>>>>>>>>>                                                     these
>>>>>>>>>>>>>>>>>>>>>>>                                                     in
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     some
>>>>>>>>>>>>>>>>>>>>>>>                                                     other
>>>>>>>>>>>>>>>>>>>>>>>                                                     way.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     How
>>>>>>>>>>>>>>>>>>>>>>>                                                     should
>>>>>>>>>>>>>>>>>>>>>>>                                                     these
>>>>>>>>>>>>>>>>>>>>>>>                                                     values
>>>>>>>>>>>>>>>>>>>>>>>                                                     be
>>>>>>>>>>>>>>>>>>>>>>>                                                     set?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     I'm
>>>>>>>>>>>>>>>>>>>>>>>                                                     also
>>>>>>>>>>>>>>>>>>>>>>>                                                     not
>>>>>>>>>>>>>>>>>>>>>>>                                                     sure
>>>>>>>>>>>>>>>>>>>>>>>                                                     what
>>>>>>>>>>>>>>>>>>>>>>>                                                     environment
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     variables/arguments
>>>>>>>>>>>>>>>>>>>>>>>                                                     are
>>>>>>>>>>>>>>>>>>>>>>>                                                     strictly
>>>>>>>>>>>>>>>>>>>>>>>                                                     necessary
>>>>>>>>>>>>>>>>>>>>>>>                                                     or
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     which
>>>>>>>>>>>>>>>>>>>>>>>                                                     are
>>>>>>>>>>>>>>>>>>>>>>>                                                     optional.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     Regards,
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     Adam
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     --
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     *Adam
>>>>>>>>>>>>>>>>>>>>>>>                                                     Hassick*
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     Senior
>>>>>>>>>>>>>>>>>>>>>>>                                                     Developer
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     UNH
>>>>>>>>>>>>>>>>>>>>>>>                                                     InterOperability
>>>>>>>>>>>>>>>>>>>>>>>                                                     Lab
>>>>>>>>>>>>>>>>>>>>>>>                                                     ahassick@iol.unh.edu
>>>>>>>>>>>>>>>>>>>>>>>                                                     <mailto:ahassick@iol.unh.edu>
>>>>>>>>>>>>>>>>>>>>>>>                                                     <mailto:ahassick@iol.unh.edu>
>>>>>>>>>>>>>>>>>>>>>>>                                                     iol.unh.edu
>>>>>>>>>>>>>>>>>>>>>>>                                                     <http://iol.unh.edu>
>>>>>>>>>>>>>>>>>>>>>>>                                                     <https://www.iol.unh.edu/>
>>>>>>>>>>>>>>>>>>>>>>>                                                     <https://www.iol.unh.edu/>
>>>>>>>>>>>>>>>>>>>>>>>                                                                        
>>>>>>>>>>>>>>>>>>>>>>>                                                     +1
>>>>>>>>>>>>>>>>>>>>>>>                                                     (603)
>>>>>>>>>>>>>>>>>>>>>>>                                                     475-8248
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     --
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     *Adam
>>>>>>>>>>>>>>>>>>>>>                                                     Hassick*
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     Senior
>>>>>>>>>>>>>>>>>>>>>                                                     Developer
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     UNH
>>>>>>>>>>>>>>>>>>>>>                                                     InterOperability
>>>>>>>>>>>>>>>>>>>>>                                                     Lab
>>>>>>>>>>>>>>>>>>>>>                                                     ahassick@iol.unh.edu
>>>>>>>>>>>>>>>>>>>>>                                                     <mailto:ahassick@iol.unh.edu>
>>>>>>>>>>>>>>>>>>>>>                                                     <mailto:ahassick@iol.unh.edu>
>>>>>>>>>>>>>>>>>>>>>                                                     iol.unh.edu
>>>>>>>>>>>>>>>>>>>>>                                                     <http://iol.unh.edu>
>>>>>>>>>>>>>>>>>>>>>                                                     <https://www.iol.unh.edu/>
>>>>>>>>>>>>>>>>>>>>>                                                     <https://www.iol.unh.edu/>
>>>>>>>>>>>>>>>>>>>>>                                                                    
>>>>>>>>>>>>>>>>>>>>>                                                     +1
>>>>>>>>>>>>>>>>>>>>>                                                     (603)
>>>>>>>>>>>>>>>>>>>>>                                                     475-8248
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>                                                                
>>>>>>>>>>>>>>>>>>>                                                     --
>>>>>>>>>>>>>>>>>>>                                                     *Adam
>>>>>>>>>>>>>>>>>>>                                                     Hassick*
>>>>>>>>>>>>>>>>>>>                                                     Senior
>>>>>>>>>>>>>>>>>>>                                                     Developer
>>>>>>>>>>>>>>>>>>>                                                     UNH
>>>>>>>>>>>>>>>>>>>                                                     InterOperability
>>>>>>>>>>>>>>>>>>>                                                     Lab
>>>>>>>>>>>>>>>>>>>                                                     ahassick@iol.unh.edu
>>>>>>>>>>>>>>>>>>>                                                     <mailto:ahassick@iol.unh.edu>
>>>>>>>>>>>>>>>>>>>                                                     <mailto:ahassick@iol.unh.edu>
>>>>>>>>>>>>>>>>>>>                                                     iol.unh.edu
>>>>>>>>>>>>>>>>>>>                                                     <http://iol.unh.edu>
>>>>>>>>>>>>>>>>>>>                                                     <https://www.iol.unh.edu/>
>>>>>>>>>>>>>>>>>>>                                                     <https://www.iol.unh.edu/>
>>>>>>>>>>>>>>>>>>>                                                                
>>>>>>>>>>>>>>>>>>>                                                     +1
>>>>>>>>>>>>>>>>>>>                                                     (603)
>>>>>>>>>>>>>>>>>>>                                                     475-8248
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>>                                                     --
>>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>>                                                     *Adam
>>>>>>>>>>>>>>>>>>>                                                     Hassick*
>>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>>                                                     Senior
>>>>>>>>>>>>>>>>>>>                                                     Developer
>>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>>                                                     UNH
>>>>>>>>>>>>>>>>>>>                                                     InterOperability
>>>>>>>>>>>>>>>>>>>                                                     Lab
>>>>>>>>>>>>>>>>>>>                                                     ahassick@iol.unh.edu
>>>>>>>>>>>>>>>>>>>                                                     <mailto:ahassick@iol.unh.edu>
>>>>>>>>>>>>>>>>>>>                                                     <mailto:ahassick@iol.unh.edu>
>>>>>>>>>>>>>>>>>>>                                                     iol.unh.edu
>>>>>>>>>>>>>>>>>>>                                                     <http://iol.unh.edu>
>>>>>>>>>>>>>>>>>>>                                                     <https://www.iol.unh.edu/>
>>>>>>>>>>>>>>>>>>>                                                     <https://www.iol.unh.edu/>
>>>>>>>>>>>>>>>>>>>                                                            
>>>>>>>>>>>>>>>>>>>                                                     +1
>>>>>>>>>>>>>>>>>>>                                                     (603)
>>>>>>>>>>>>>>>>>>>                                                     475-8248
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>>                                                     --
>>>>>>>>>>>>>>>>>>                                                     *Adam
>>>>>>>>>>>>>>>>>>                                                     Hassick*
>>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>>                                                     Senior
>>>>>>>>>>>>>>>>>>                                                     Developer
>>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>>                                                     UNH
>>>>>>>>>>>>>>>>>>                                                     InterOperability
>>>>>>>>>>>>>>>>>>                                                     Lab
>>>>>>>>>>>>>>>>>>                                                     ahassick@iol.unh.edu
>>>>>>>>>>>>>>>>>>                                                     <mailto:ahassick@iol.unh.edu>
>>>>>>>>>>>>>>>>>>                                                     <mailto:ahassick@iol.unh.edu>
>>>>>>>>>>>>>>>>>>                                                     iol.unh.edu
>>>>>>>>>>>>>>>>>>                                                     <http://iol.unh.edu>
>>>>>>>>>>>>>>>>>>                                                     <https://www.iol.unh.edu/>
>>>>>>>>>>>>>>>>>>                                                     <https://www.iol.unh.edu/>
>>>>>>>>>>>>>>>>>>                                                        
>>>>>>>>>>>>>>>>>>                                                     +1
>>>>>>>>>>>>>>>>>>                                                     (603)
>>>>>>>>>>>>>>>>>>                                                     475-8248
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>                                                     -- 
>>>>>>>>>>>>>>>>>                                                     *Adam
>>>>>>>>>>>>>>>>>                                                     Hassick*
>>>>>>>>>>>>>>>>>                                                     Senior
>>>>>>>>>>>>>>>>>                                                     Developer
>>>>>>>>>>>>>>>>>                                                     UNH
>>>>>>>>>>>>>>>>>                                                     InterOperability
>>>>>>>>>>>>>>>>>                                                     Lab
>>>>>>>>>>>>>>>>>                                                     ahassick@iol.unh.edu
>>>>>>>>>>>>>>>>>                                                     <mailto:ahassick@iol.unh.edu>
>>>>>>>>>>>>>>>>>                                                     <mailto:ahassick@iol.unh.edu>
>>>>>>>>>>>>>>>>>                                                     iol.unh.edu
>>>>>>>>>>>>>>>>>                                                     <http://iol.unh.edu>
>>>>>>>>>>>>>>>>>                                                     <https://www.iol.unh.edu/>
>>>>>>>>>>>>>>>>>                                                     <https://www.iol.unh.edu/>
>>>>>>>>>>>>>>>>>                                                     +1
>>>>>>>>>>>>>>>>>                                                     (603)
>>>>>>>>>>>>>>>>>                                                     475-8248
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>                                                 --
>>>>>>>>>>>>>>>                                                 *Adam
>>>>>>>>>>>>>>>                                                 Hassick*
>>>>>>>>>>>>>>>                                                 Senior
>>>>>>>>>>>>>>>                                                 Developer
>>>>>>>>>>>>>>>                                                 UNH
>>>>>>>>>>>>>>>                                                 InterOperability
>>>>>>>>>>>>>>>                                                 Lab
>>>>>>>>>>>>>>>                                                 ahassick@iol.unh.edu
>>>>>>>>>>>>>>>                                                 iol.unh.edu
>>>>>>>>>>>>>>>                                                 <https://www.iol.unh.edu/>
>>>>>>>>>>>>>>>                                                 +1 (603)
>>>>>>>>>>>>>>>                                                 475-8248
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>                                             --
>>>>>>>>>>>>>                                             *Adam Hassick*
>>>>>>>>>>>>>                                             Senior Developer
>>>>>>>>>>>>>                                             UNH
>>>>>>>>>>>>>                                             InterOperability
>>>>>>>>>>>>>                                             Lab
>>>>>>>>>>>>>                                             ahassick@iol.unh.edu
>>>>>>>>>>>>>                                             iol.unh.edu
>>>>>>>>>>>>>                                             <https://www.iol.unh.edu/>
>>>>>>>>>>>>>                                             +1 (603) 475-8248
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                                         --
>>>>>>>>>>>                                         *Adam Hassick*
>>>>>>>>>>>                                         Senior Developer
>>>>>>>>>>>                                         UNH InterOperability Lab
>>>>>>>>>>>                                         ahassick@iol.unh.edu
>>>>>>>>>>>                                         iol.unh.edu
>>>>>>>>>>>                                         <https://www.iol.unh.edu/>
>>>>>>>>>>>                                         +1 (603) 475-8248
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                                 --
>>>>>>>>>                                 *Adam Hassick*
>>>>>>>>>                                 Senior Developer
>>>>>>>>>                                 UNH InterOperability Lab
>>>>>>>>>                                 ahassick@iol.unh.edu
>>>>>>>>>                                 iol.unh.edu
>>>>>>>>>                                 <https://www.iol.unh.edu/>
>>>>>>>>>                                 +1 (603) 475-8248
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                             --
>>>>>>>>                             *Adam Hassick*
>>>>>>>>                             Senior Developer
>>>>>>>>                             UNH InterOperability Lab
>>>>>>>>                             ahassick@iol.unh.edu
>>>>>>>>                             iol.unh.edu <https://www.iol.unh.edu/>
>>>>>>>>                             +1 (603) 475-8248
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                         --
>>>>>>>>                         *Adam Hassick*
>>>>>>>>                         Senior Developer
>>>>>>>>                         UNH InterOperability Lab
>>>>>>>>                         ahassick@iol.unh.edu
>>>>>>>>                         iol.unh.edu <https://www.iol.unh.edu/>
>>>>>>>>                         +1 (603) 475-8248
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                 -- 
>>>>>                 *Adam Hassick*
>>>>>                 Senior Developer
>>>>>                 UNH InterOperability Lab
>>>>>                 ahassick@iol.unh.edu
>>>>>                 iol.unh.edu <https://www.iol.unh.edu/>
>>>>>                 +1 (603) 475-8248
>>>>
>>>>
>>>>
>>>>             -- 
>>>>             *Adam Hassick*
>>>>             Senior Developer
>>>>             UNH InterOperability Lab
>>>>             ahassick@iol.unh.edu
>>>>             iol.unh.edu <https://www.iol.unh.edu/>
>>>>             +1 (603) 475-8248
>>>
>>>
>>>
>>>         -- 
>>>         *Adam Hassick*
>>>         Senior Developer
>>>         UNH InterOperability Lab
>>>         ahassick@iol.unh.edu
>>>         iol.unh.edu <https://www.iol.unh.edu/>
>>>         +1 (603) 475-8248
>>
>

[-- Attachment #2: Type: text/html, Size: 256336 bytes --]

  reply	other threads:[~2023-10-27  8:14 UTC|newest]

Thread overview: 51+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CAC-YWqiQfH4Rx-Et1jGHhGK9i47d0AArKy-B2P77iYbbM+Lpig@mail.gmail.com>
     [not found] ` <C3B08390-DA6D-4BDC-BBD7-98561F92FE33@oktetlabs.ru>
     [not found]   ` <35340484-1d7e-7e5f-cad4-c965ba541397@oktetlabs.ru>
2023-08-17 17:03     ` Adam Hassick
2023-08-18 18:40       ` Andrew Rybchenko
2023-08-20  8:40         ` Andrew Rybchenko
2023-08-21 14:19           ` Adam Hassick
2023-08-23 14:45             ` Adam Hassick
2023-08-24  8:22               ` Andrew Rybchenko
2023-08-24 14:30                 ` Adam Hassick
2023-08-24 18:34                   ` Andrew Rybchenko
2023-08-24 20:29                     ` Adam Hassick
2023-08-24 20:54                       ` Andrew Rybchenko
2023-08-25 13:57                         ` Andrew Rybchenko
2023-08-25 14:06                           ` Adam Hassick
2023-08-25 14:41                             ` Andrew Rybchenko
2023-08-25 17:35                               ` Andrew Rybchenko
2023-08-28 15:02                                 ` Adam Hassick
2023-08-28 21:05                                   ` Andrew Rybchenko
2023-08-29 12:07                                     ` Andrew Rybchenko
2023-08-29 14:02                                       ` Adam Hassick
2023-08-29 20:43                                         ` Andrew Rybchenko
2023-08-31 19:38                                           ` Adam Hassick
2023-09-01  7:59                                             ` Andrew Rybchenko
2023-09-05 15:01                                               ` Adam Hassick
2023-09-06 11:36                                                 ` Andrew Rybchenko
2023-09-06 15:00                                                   ` Adam Hassick
2023-09-08 14:57                                                     ` Adam Hassick
2023-09-13 15:45                                                       ` Andrew Rybchenko
2023-09-18  6:15                                                         ` Andrew Rybchenko
2023-09-18  6:23                                                           ` Konstantin Ushakov
2023-09-18  6:26                                                             ` Andrew Rybchenko
2023-09-18 14:44                                                               ` Adam Hassick
2023-09-18 15:04                                                                 ` Andrew Rybchenko
2023-10-04 13:48                                                                   ` Adam Hassick
2023-10-05 10:25                                                                     ` Andrew Rybchenko
2023-10-10 14:09                                                                       ` Adam Hassick
2023-10-11 11:46                                                                         ` Andrew Rybchenko
2023-10-23 11:11                                                                         ` Andrew Rybchenko
2023-10-25 20:27                                                                           ` Adam Hassick
2023-10-26 12:19                                                                             ` Andrew Rybchenko
2023-10-26 17:44                                                                               ` Adam Hassick
2023-10-27  8:01                                                                                 ` Andrew Rybchenko [this message]
2023-10-27 19:13                                                                                 ` Andrew Rybchenko
2023-11-06 23:16                                                                                   ` Adam Hassick
2023-11-07 16:57                                                                                     ` Andrew Rybchenko
2023-11-07 20:30                                                                                       ` Adam Hassick
2023-11-08  7:20                                                                                         ` Andrew Rybchenko
2023-11-16 20:03                                                                                           ` Adam Hassick
2023-11-16 20:38                                                                                             ` DPDK Coverity test run Mcnamara, John
2023-11-16 20:43                                                                                               ` Patrick Robb
2023-11-16 20:56                                                                                                 ` Mcnamara, John
2023-11-20 17:18                                                                                             ` Setting up DPDK PMD Test Suite Andrew Rybchenko
2023-12-01 14:39                                                                                               ` Andrew Rybchenko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=f9864f9f-d9d0-4992-afd6-a07d0047daad@oktetlabs.ru \
    --to=andrew.rybchenko@oktetlabs.ru \
    --cc=Konstantin.Ushakov@oktetlabs.ru \
    --cc=ahassick@iol.unh.edu \
    --cc=ci@dpdk.org \
    --cc=probb@iol.unh.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).