From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5EB3645A68; Mon, 30 Sep 2024 15:43:20 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4CFC5402A3; Mon, 30 Sep 2024 15:43:20 +0200 (CEST) Received: from mail-ed1-f49.google.com (mail-ed1-f49.google.com [209.85.208.49]) by mails.dpdk.org (Postfix) with ESMTP id 71E8D4014F for <dev@dpdk.org>; Mon, 30 Sep 2024 15:43:19 +0200 (CEST) Received: by mail-ed1-f49.google.com with SMTP id 4fb4d7f45d1cf-5c89e66012aso949966a12.2 for <dev@dpdk.org>; Mon, 30 Sep 2024 06:43:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pantheon.tech; s=google; t=1727703799; x=1728308599; darn=dpdk.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=zA/nUIR1ZPofDTDLMUa1vxDNyItJ4GmyxTyw2s8LWXI=; b=fzdjh5p6o/avtnq16FLEleFAmCEWm6ygNqPzn2kiweLDDWuUHMIn9sZijmAqx8lbsJ g5ysRAKKmTbV8fFRed4WUEm2qLrw6HXyDLK8WqGa0zh3lpryWGwpClsh42kxp/ASOoNt JZwLJa4ph61W/J27uIf5jOOwWOMvL0qIxU27XtDCY5sdG/90n8ZnPK/hyI8YX3ASL4UT jCf2/YXRC2oeaawsmNr4nJP6aEzwHr0xg/QKim8uqe2pgJfHtlWEQBiu2ylYt+hdti83 juHcBew9CEUFRly7/xYjhcq9nMx39pH4ypizFVpNM/BQn8yYndTmlOumBG2EgwB5d+ac 7pgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727703799; x=1728308599; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=zA/nUIR1ZPofDTDLMUa1vxDNyItJ4GmyxTyw2s8LWXI=; b=aso3wZQyfxL5t9B2k7JGztn+7MrdfZYnmkc6mUPC8emAjvT07I14UGPehTJGK09CHq yPyl3r/NzreHAU7NQC+gc2rKoPZ578B2OSSslRD2Npw7zVtz+OvOxvUGQDAJwBHtTcaO /36R/ApXtzXp8LAnW+GBsUBslLJ6p5C+RE9b4PYx20Je6Wf+l9KyrU46C31Ltgo5CsHT 0/nMA7O8OQCdLbks3lXev8A21013NNVCkeZxeLnAyAuTkw9Sco2IM/JttMm2U+ModRzk XVh6CBvYcGXKu6wNKCNULi9BRXGJYyzWyN2SNlnhRkfK+4Xw4TjN0YnI4XKiEfNAqE4N 3ldA== X-Gm-Message-State: AOJu0Yyn3iAUIFHBoLsicsSvq1dEtMGhPOFRwpqCO4phg5Uy/aYkfiB8 bJz9SlIRaXyEQHwTiWSIXZxT5C1F0XZLfx1Ki6poQM04+45S6ldIaPFc5Asb9bc= X-Google-Smtp-Source: AGHT+IEOPUWSHvqfKmKb2zRqq34FsEg/WOhMQZJkY03J+9H766sTjIBeJUKJp494dJupOBgobs/KMQ== X-Received: by 2002:a05:6402:34cc:b0:5c2:4cbe:ac1c with SMTP id 4fb4d7f45d1cf-5c8824cde60mr9407690a12.4.1727703798910; Mon, 30 Sep 2024 06:43:18 -0700 (PDT) Received: from [192.168.200.22] ([84.245.121.62]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88249540bsm4488763a12.96.2024.09.30.06.43.18 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 30 Sep 2024 06:43:18 -0700 (PDT) Message-ID: <33cadf61-b3cd-4218-ac1f-941c4d37d1a6@pantheon.tech> Date: Mon, 30 Sep 2024 15:43:17 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v4 00/11] dts: add test skipping based on capabilities To: thomas@monjalon.net, Honnappa.Nagarahalli@arm.com, paul.szczepanek@arm.com, Luca.Vizzarro@arm.com, alex.chapman@arm.com, probb@iol.unh.edu, jspewock@iol.unh.edu, npratte@iol.unh.edu, dmarx@iol.unh.edu Cc: dev@dpdk.org References: <20240301155416.96960-1-juraj.linkes@pantheon.tech> <20240924082008.14763-1-juraj.linkes@pantheon.tech> Content-Language: en-US From: =?UTF-8?Q?Juraj_Linke=C5=A1?= <juraj.linkes@pantheon.tech> In-Reply-To: <20240924082008.14763-1-juraj.linkes@pantheon.tech> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org On 24. 9. 2024 10:20, Juraj Linkeš wrote: > Add an automated way to gather available capabilities of the tested > hardware and skip test suites or cases which require capabilities that > are not available. > > This is done through two decorators: > 1. The first marks a test suite method as test case. This populates the > default attributes of each test case. > 2. The seconds adds the required capabilities to a test suite or case, > using the attributes from 1). > > Two types of capabilities are added: > 1. NIC capabilities. These are gathered once DPDK has been built because > we use testpmd for this. It's possible to add a function that will > add configuration before assessing capabilities associated with the > function. This is because some capabilities return different status > with different configuration present. > 2. The topology capability. Each test case is marked as requiring a > default topology. The required topology of a test case (or a whole > test suite) may be change with the second decorator. > > This is how it all works: > 1. The required capabilities are first all gathered from all test suites > and test cases. > 2. The list of required capabilities is divided into supported and > unsupported capabilities. In this step, the probing of hardware > and/or anything else that needs to happen to gauge whether a > capability is supported is done. > 3. Each test suite and test case is then marked to be skipped if any of > their required capabilities are not supported. > > v4: > Rebased on next-dts. > Adjusted the decorator for setting and reverting 9000 MTU to be > parametrized. > Added references to DPDK libraries and testpmd code to testpmd command > parsing methods. > Updated capability docstrings. Moved the capability decorator to > NicCapability. > > Juraj Linkeš (11): > dts: add the aenum dependency > dts: add test case decorators > dts: add mechanism to skip test cases or suites > dts: add support for simpler topologies > dts: add basic capability support > dts: add NIC capability support > dts: add NIC capabilities from show rxq info > dts: add topology capability > doc: add DTS capability doc sources > dts: add Rx offload capabilities > dts: add NIC capabilities from show port info > > .../framework.testbed_model.capability.rst | 6 + > doc/api/dts/framework.testbed_model.rst | 2 + > .../dts/framework.testbed_model.topology.rst | 6 + > dts/framework/remote_session/testpmd_shell.py | 461 +++++++++++++++- > dts/framework/runner.py | 155 +++--- > dts/framework/test_result.py | 119 ++-- > dts/framework/test_suite.py | 161 +++++- > dts/framework/testbed_model/capability.py | 507 ++++++++++++++++++ > dts/framework/testbed_model/node.py | 2 +- > dts/framework/testbed_model/port.py | 4 +- > dts/framework/testbed_model/topology.py | 127 +++++ > dts/poetry.lock | 14 +- > dts/pyproject.toml | 1 + > dts/tests/TestSuite_hello_world.py | 10 +- > dts/tests/TestSuite_os_udp.py | 3 +- > dts/tests/TestSuite_pmd_buffer_scatter.py | 14 +- > dts/tests/TestSuite_smoke_tests.py | 8 +- > 17 files changed, 1447 insertions(+), 153 deletions(-) > create mode 100644 doc/api/dts/framework.testbed_model.capability.rst > create mode 100644 doc/api/dts/framework.testbed_model.topology.rst > create mode 100644 dts/framework/testbed_model/capability.py > create mode 100644 dts/framework/testbed_model/topology.py > Applied to next-dts, thanks for the reviews.