From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9DBBE438A3; Fri, 12 Jan 2024 14:25:09 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 372244026E; Fri, 12 Jan 2024 14:25:09 +0100 (CET) Received: from mail-ed1-f48.google.com (mail-ed1-f48.google.com [209.85.208.48]) by mails.dpdk.org (Postfix) with ESMTP id 1141B4025E for ; Fri, 12 Jan 2024 14:25:08 +0100 (CET) Received: by mail-ed1-f48.google.com with SMTP id 4fb4d7f45d1cf-554fe147ddeso7336063a12.3 for ; Fri, 12 Jan 2024 05:25:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pantheon.tech; s=google; t=1705065907; x=1705670707; darn=dpdk.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=U9yrh0j4kiEK77Bcx5Lcz9DdTFWHFwAAB6vDuQaHd1w=; b=R/4ALo/G02Mt623wM6CWzjQOexvsXCsYKbxHk9W0o8wqZEJPuPttArtM5oOdJh5Fho EalZyk6uApbf9iu07Lpug/vy7R71ajjMd72316LZcHOAmkzUkg5qqzdYN2H6AaKeKVri 2KAKeXfR/7VNmEXSqEqVVroP07KtMOBrQkRvgUDQkm8WthW4sh7AyxO257LEG4DNsERi ZLqt5TpfRovcoGekDh9XMC95r92NEgE+RZuMD4Yxk4D8FKfF2m7KNyf7lx6CzXlP8aOe m58vP6tcuyuRkVIuARJd0PgZNCCxoiJQZsLO+kx+df/M2AXg+0euRzhhj71bVtKmH/VY 97+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705065907; x=1705670707; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U9yrh0j4kiEK77Bcx5Lcz9DdTFWHFwAAB6vDuQaHd1w=; b=f/liRoY8MlK4Jfj5DjdhzCulFGMbSV+6PbCQeYE1aCE3Ocxnqd4AvnyvR8JqD2dNSW EhFPSjWS4B/M5f9Lp0c34bmlpy9sWR+Gawaaj+OFDEw1WfjjhricKXtadFBvQJ04jDbZ /WQ94X2y4Hz8/KvaMPDM1UVoCkxCQWZQ+jJhyRVgv9+vncVON1nT2F7373OZV1RvHfy7 zQjfdnuX7H7rPEvSkTTA0zo4cR7fsWQbo33OzPa0pKZIu7ubAGCPVmidFgk22+L3fcUd yigzt9Odl8FkdycV81yIL0XRNci3zmfsjbH+vWxmN4zdUqZ5gH5TShJ1s8+Oi3vS3YM+ A58A== X-Gm-Message-State: AOJu0Yx8iK7+g9ehpVpe2NQ6+gCITZzkVViFkqr+KTvRLMMvXjPk68bo myLHzui3PuHzGC+bRHRwscFZbma+TIh6F3h2FirgXvPVlm+edvf5VMmpSqcRh8E9/Q== X-Google-Smtp-Source: AGHT+IHznw5ygeadXlu3fK6ORN9iTt9GV0Pyoy0EKP9tHL/jFgpE0xbbPf7ruSJpR756doKSEXkbJ+erIdMP+WpTGZw= X-Received: by 2002:a17:906:2b45:b0:a2a:ddd1:aeaa with SMTP id b5-20020a1709062b4500b00a2addd1aeaamr311609ejg.276.1705065907574; Fri, 12 Jan 2024 05:25:07 -0800 (PST) MIME-Version: 1.0 References: <20240103125438.182098-1-Luca.Vizzarro@arm.com> In-Reply-To: <20240103125438.182098-1-Luca.Vizzarro@arm.com> From: =?UTF-8?Q?Juraj_Linke=C5=A1?= Date: Fri, 12 Jan 2024 14:24:56 +0100 Message-ID: Subject: Re: [PATCH] dts: improve documentation To: Luca Vizzarro Cc: dev@dpdk.org, Paul Szczepanek , Thomas Monjalon , Lijuan Tu Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org I have two extra suggestions apart from the comments below: There's a typo inside the "How To Write a Test Suite" section: In that case, nothing will happen when they're is executed. And Mypy is missing from the list of linters in the "DTS Developer Tools" section, could you please add it? > diff --git a/doc/guides/tools/dts.rst b/doc/guides/tools/dts.rst > index 32c18ee472..31495cad51 100644 > --- a/doc/guides/tools/dts.rst > +++ b/doc/guides/tools/dts.rst > @@ -335,3 +336,183 @@ There are three tools used in DTS to help with code= checking, style and formatti > These three tools are all used in ``devtools/dts-check-format.sh``, > the DTS code check and format script. > Refer to the script for usage: ``devtools/dts-check-format.sh -h``. > + > +Configuration Schema > +-------------------- Just out of curiosity, is this generated from the schema? It's a pretty neat format, but maintaining it could be a nightmare without a script that would always produce the same format. > + > +Definitions > +~~~~~~~~~~~ > + The section names look to be taken from the schema and all of the terminology is taken from the schema. Would it make sense to use YAML terminology here, since people will try to use this information in YAML files? Or maybe explain what we mean by definitions, properties, objects, arrays or maybe some other things which YAML doesn't specify. > +_`Node name` > + *string* =E2=80=93 A unique identifier for a node. **Examples**: ``SU= T1``, ``TG1``. > + > +_`ARCH` > + *string* =E2=80=93 The CPU architecture. **Supported values**: ``x86_= 64``, ``arm64``, ``ppc64le``. > + > +_`CPU` > + *string* =E2=80=93 The CPU microarchitecture. Use ``native`` for x86.= **Supported values**: ``native``, ``armv8a``, ``dpaa2``, ``thunderx``, ``x= gene1``. > + > +_`OS` > + *string* =E2=80=93 The operating system. **Supported values**: ``linu= x``. > + > +_`Compiler` > + *string* =E2=80=93 The compiler used for building DPDK. **Supported v= alues**: ``gcc``, ``clang``, ``icc``, ``mscv``. > + > +_`Build target` > + *object* =E2=80=93 Build targets supported by DTS for building DPDK, = described as: > + > + =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > + ``arch`` See `ARCH`_ > + ``os`` See `OS`_ > + ``cpu`` See `CPU`_ > + ``compiler`` See `Compiler`_ > + ``compiler_wrapper`` *string* =E2=80=93 Value prepended to the CC var= iable for the DPDK build. > + > + **Example**: ``ccache`` > + =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > + > +_`hugepages` > + *object* =E2=80=93 hugepages described as: > + > + =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > + ``amount`` *integer* =E2=80=93 The amount of hugepages to c= onfigure. > + We should update "amount" (uncountable) to something that's countable, such as count or number. > + Hugepage size will be the system default. > + ``force_first_numa`` (*optional*, defaults to ``false``) =E2=80=93 If= ``true``, it forces the > + > + configuration of hugepages on the first NUMA nod= e. > + =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > + > +_`Network port` > + *object* =E2=80=93 the NIC port described as: > + > + =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D > + ``pci`` *string* =E2=80=93 the local PCI address of th= e port. **Example**: ``0000:00:08.0`` > + ``os_driver_for_dpdk`` | *string* =E2=80=93 this port's device driver= when using with DPDK > + | When setting up the SUT, DTS will bind the n= etwork device to this driver > + | for compatibility with DPDK. > + > + **Examples**: ``vfio-pci``, ``mlx5_core`` > + ``os_driver`` | *string* =E2=80=93 this port's device driver= when **not** using with DPDK > + | When tearing down the tests on the SUT, DTS = will bind the network device > + | *back* to this driver. This driver is meant = to be the one that the SUT would > + | normally use for this device, or whichever d= river it is preferred to leave the > + | device bound to after testing. > + Of note here is that some traffic generators (to which the port config also applies) won't be using os_driver_for_dpdk (such as Scapy), but rather os_driver, so the use is broader. > + **Examples**: ``i40e``, ``mlx5_core`` > + ``peer_node`` *string* =E2=80=93 the name of the peer node c= onnected to this port. > + ``peer_pci`` *string* =E2=80=93 the PCI address of the peer= node port. **Example**: ``000a:01:00.1`` > + =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D > +Example > +~~~~~~~ > + > +The following example (which can be found in ``dts/conf.yaml``) sets up = two nodes: > + > +* ``SUT1`` which is already setup with the DPDK build requirements and a= ny other > + required for execution; > +* ``TG1`` which already has Scapy installed in the system. > + > +And they both have two network ports which are physically connected to e= ach other. > + > +.. note:: > + This example assumes that you have setup SSH keys in both the system = under test > + and traffic generator nodes. > + > +.. literalinclude:: ../../../dts/conf.yaml > + :language: yaml > + :start-at: executions: > \ No newline at end of file The last newline is missing. > diff --git a/dts/conf.yaml b/dts/conf.yaml > index 37967daea0..2d6fa38a2c 100644 > --- a/dts/conf.yaml > +++ b/dts/conf.yaml > @@ -1,65 +1,76 @@ > # SPDX-License-Identifier: BSD-3-Clause > # Copyright 2022-2023 The DPDK contributors > +# Copyright 2023 Arm Limited > > executions: > + # define one execution environment > - build_targets: > - arch: x86_64 > os: linux > cpu: native > + # the combination of the following two makes CC=3D"ccache gcc" > compiler: gcc > compiler_wrapper: ccache > - perf: false > - func: true > - skip_smoke_tests: false # optional flag that allows you to skip smok= e tests > - test_suites: > + perf: false # disable performance testing > + func: true # enable functional testing > + skip_smoke_tests: false Let's keep the note that the skip_smoke_tests flag is optional > + test_suites: # the following test suites will be run in their entire= ty > - hello_world > - os_udp > + # The machine running the DPDK test executable > system_under_test_node: > node_name: "SUT 1" > vdevs: # optional; if removed, vdevs won't be used in the executio= n > - "crypto_openssl" > + # Traffic generator node to use for this execution environment > traffic_generator_node: "TG 1" > nodes: > + # Define a system under test node, having two network ports physically > + # connected to the corresponding ports in TG 1 (the peer node) > - name: "SUT 1" > hostname: sut1.change.me.localhost > user: dtsuser > arch: x86_64 > os: linux > - lcores: "" > - use_first_core: false > - memory_channels: 4 > + lcores: "" # use all the available logical cores > + use_first_core: false # tells DPDK to use any physical core > + memory_channels: 4 # tells DPDK to use 4 memory channels > hugepages: # optional; if removed, will use system hugepage configu= ration > amount: 256 > force_first_numa: false > ports: > + # sets up the physical link between "SUT 1"@000:00:08.0 and "TG 1"= @0000:00:08.0 > - pci: "0000:00:08.0" > os_driver_for_dpdk: vfio-pci # OS driver that DPDK will use > - os_driver: i40e > + os_driver: i40e # OS driver to bind when the tests = are not running > peer_node: "TG 1" > peer_pci: "0000:00:08.0" > + # sets up the physical link between "SUT 1"@000:00:08.1 and "TG 1"= @0000:00:08.1 > - pci: "0000:00:08.1" > os_driver_for_dpdk: vfio-pci > os_driver: i40e > peer_node: "TG 1" > peer_pci: "0000:00:08.1" > + # Define a Scapy traffic generator node, having two network ports > + # physically connected to the corresponding ports in SUT 1 (the peer n= ode). > - name: "TG 1" > hostname: tg1.change.me.localhost > user: dtsuser > arch: x86_64 > os: linux > - lcores: "" The traffic generator may need this core configuration. However, since it's not required for all traffic generators (such as Scapy), we could just move to the traffic_generator section. That would require some code modifications though, but even the removal of lcores and use_first_core should be addressed in the configuration classes as well. Have you tried running DTS with these changes? > ports: > + # sets up the physical link between "TG 1"@000:00:08.0 and "SUT 1"= @0000:00:08.0 > - pci: "0000:00:08.0" > os_driver_for_dpdk: rdma > os_driver: rdma > peer_node: "SUT 1" > peer_pci: "0000:00:08.0" > + # sets up the physical link between "SUT 1"@000:00:08.0 and "TG 1"= @0000:00:08.0 > - pci: "0000:00:08.1" > os_driver_for_dpdk: rdma > os_driver: rdma > peer_node: "SUT 1" > peer_pci: "0000:00:08.1" > - use_first_core: false > hugepages: # optional; if removed, will use system hugepage configu= ration > amount: 256 > force_first_numa: false > -- > 2.34.1 >