From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from wes1-so2.wedos.net (wes1-so2.wedos.net [46.28.106.16]) by dpdk.org (Postfix) with ESMTP id 767EF5585 for ; Fri, 29 Apr 2016 22:52:15 +0200 (CEST) Received: from jvn (dynamic-109-81-211-244.ipv4.broadband.iol.cz [109.81.211.244]) by wes1-so2.wedos.net (Postfix) with ESMTPSA id 3qxQqf6kSPzWx; Fri, 29 Apr 2016 22:52:14 +0200 (CEST) Date: Fri, 29 Apr 2016 22:52:19 +0200 From: Jan Viktorin To: Bruce Richardson Cc: dev@dpdk.org, Thomas Monjalon , David Marchand Message-ID: <20160429225219.46d23129@jvn> In-Reply-To: <20160429144226.GA20192@bricha3-MOBL3> References: <1461935496-20367-1-git-send-email-viktorin@rehivetech.com> <20160429144226.GA20192@bricha3-MOBL3> Organization: RehiveTech X-Mailer: Claws Mail 3.13.0 (GTK+ 2.24.28; x86_64-unknown-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [RFC 0/4] Include resources in tests X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Apr 2016 20:52:15 -0000 On Fri, 29 Apr 2016 15:42:27 +0100 Bruce Richardson wrote: > On Fri, Apr 29, 2016 at 03:11:32PM +0200, Jan Viktorin wrote: > > Hello, > > > > this patch set introduces a mechanism to include a resource (in general a blob) > > into the test binary. This allows to make tests less dependent on the target > > testing environment. The first use case is testing of PCI bus scan by changing > > the hard-coded path (/sys/bus/pci/devices) to something different and provide > > a fake tree of devices with the test. It can help with testing of device-tree > > parsing as I've proposed in [1] where such mechanism was missing at that time. > > I'd like to use such framework for the SoC infra testing as well. > > > > The patch set introduces a struct resource into the app/test. The resource is > > generic to include any kind of binary data. The binary data can be created in > > C or linked as an object file (created by objcopy). I am not sure where to > > place the objcopy logic and how to perform guessing of the objcopy arguments > > as they are pretty non-standard. > > > > To include a complex resource (a file hierarchy), the last patch implements > > an archive extraction logic. So, it is possible to include a tar archive and > > unpack it before a test starts. Any ideas how to do this in a better way are > > welcome. > > > > [1] http://comments.gmane.org/gmane.comp.networking.dpdk.devel/36545 > > > > Regards > > Jan Viktorin > > > > Hi Jan, > > this looks really interesting, especially since just yesterday I was looking at > taking the million-entry lpm test routing table out of the C code and into a > separate resource file in this case an ini file. If it is an automatic test case which can show some regressions over time then it is a good example of the resource usage. > > In terms of a solution, I'm not convinced of the placing of the blobs inside the > test binary. I think a better solution would be to allow the different autotests > to take parameters from the commandline, so that the user can specify the path > to the file to use for the test. What would be your opinion of such a scheme? I think that we are already at this stage. You can read parameters e.g. from the environment vars (so in fact, no changes to the DPDK testing code are needed). The thing is that I want to say "run tests" and don't care about any parameters to receive the result. It might be possible to pass parameters to the tests (optionally) to reuse the testing code base for different cases. However, initially, the test (in my opinion) should run on any architecture on any device without any configuration and return consistent results. How can I test probing of PCI devices on my PC when I have different network card then the author of the test? I cannot or I have to pass parameters which is not what I want as it complicates every single such test (and I have to understand those tests or understand the parameters syntax even though the code is unrelated to my work). I just want to check that the DPDK works as expected - without regressions. And what about the case when I have no such card (running automatic tests on a build server)? I don't like including the resources in binaries but I didn't find any better way yet. I need to easily install the tests together with the testing data (resources) to an embedded device and perform the tests on the target system. When you are testing x86 code on your x86 you don't have to care too much about the location of resources. But I do. And there is no standard way (at the moment) how to install the resources together with the testing code. What I like on this solution is that the DPDK git repository would contain the testing data as a real file hierarchy (e.g. the fake /sys/bus/...). After build and install of DPDK, the file hierachy is packed (and compressed if needed) together with the tests. So moving to the target platform works without any other changes to the testing/install infrastructure. I could potentially run PCI tests on my ARM board without any PCI available and don't have to ignore the failures of those tests (as they simply pass). Regards Jan > > /Bruce -- Jan Viktorin E-mail: Viktorin@RehiveTech.com System Architect Web: www.RehiveTech.com RehiveTech Brno, Czech Republic