From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 591E0A0C47; Wed, 11 Aug 2021 16:08:46 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3AC07410EC; Wed, 11 Aug 2021 16:08:46 +0200 (CEST) Received: from mail-oi1-f176.google.com (mail-oi1-f176.google.com [209.85.167.176]) by mails.dpdk.org (Postfix) with ESMTP id 7307D40C35 for ; Wed, 11 Aug 2021 16:08:45 +0200 (CEST) Received: by mail-oi1-f176.google.com with SMTP id h11so4820839oie.0 for ; Wed, 11 Aug 2021 07:08:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=O28qTlCcvy4RNCWtNKK+AQ7KqHbewRWkk2+fXiv6yO8=; b=gwgUfar2ile4/DZQVThxCgfPHyW3dFTq/SRj6RLTBWyeJZkc+HUYNq1wIgOSfpGtoA 5y9FVB39tv2egWOPWPtRttPh1Z2nc/jTuefVbzzIUJD7VnsdsfT8L2JmqTQSAMTtK0d5 jXvw4B9h9UaUs+kBm1c9qwcKYYSZrgOXmj1U8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=O28qTlCcvy4RNCWtNKK+AQ7KqHbewRWkk2+fXiv6yO8=; b=e5Ruol7SX4NRRBpEd4gjJKRGMUepS8UUcFESBWU4dYhc42y++w28QoRHYwcPkHDI2+ qr4j/HhxoAM40akOgMbFJgOJ/XgvuFdxEwmPs0YKy6V+3ZYpUUbRncFztPPGNz2R5jbo iV1wzvkzjeHsBP7z+GKIdxQMsph8b/ojETl7GD0QM3Ug/zV7rZWUAArfQnZv+7FU+/iN ANR1VKuvGp3yHB5XwlX/uSYEwfzcX807oL18VLXOpL53s5JsWqoQqLOIlnc113ejgoet q7WFAjUUx4ANjc5RgpnDVJMzfB18a+IgH0IT5r5uZoylHOJS5byCU9WfUvTY5GOPpMV2 yMLg== X-Gm-Message-State: AOAM533lq8OLnBqIRqz1HAVe52QPYLCkXJu97VvB+T6lY6VtHlCN5rve g1mp8gz0CWHc6QXW8277TtSgdF8xjLxTWi8PLMXuctHteQU= X-Google-Smtp-Source: ABdhPJwiH6a/NNg/XoBesav/FDZ8kCP2TgMmARyMSylHGIoTiTdahVWn0AKDH3+sdkLwis6r7IavH7Wa5a1+mt4+6+E= X-Received: by 2002:a05:6808:1d1:: with SMTP id x17mr7348766oic.69.1628690924605; Wed, 11 Aug 2021 07:08:44 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Owen Hilyard Date: Wed, 11 Aug 2021 10:08:08 -0400 Message-ID: To: dts@dpdk.org Cc: =?UTF-8?Q?Juraj_Linke=C5=A1?= , Honnappa.Nagarahalli@arm.com, "Tu, Lijuan" Content-Type: multipart/alternative; boundary="00000000000002aa1905c94928f3" Subject: Re: [dts] [RFC] Reorganization of DTS X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" --00000000000002aa1905c94928f3 Content-Type: text/plain; charset="UTF-8" Hello everyone, Since there was no feedback for a week, the DTS working group has decided to move ahead with this proposal as written. Owen Hilyard On Thu, Aug 5, 2021 at 2:49 PM Owen Hilyard wrote: > Hello everyone, > > The DTS working group has decided that, conditional upon no issues being > discovered, DTS should be re-organized into a more standard python > project format. We have a few reasons for this. > > How DTS currently is structured is only possible due to modifications of > the python interpreter search path. The specific modifications are adding > all of the directories with python code in them, such as framework, nic, > and tests, to the search path. This enables an import statement like > "import logging" to work in a test suite, as opposed to the "import > framework.logging as logging" that would otherwise be needed. However, this > change is not detected by most python tooling. As a result, features like > autocompletion and jump-to-definition are broken in many IDEs, and static > analysis tools can only do surface-level inspections, which also produce a > lot of false positives due to undefined variables. There are two core > reasons the DTS working group has decided to move in this direction. > > 1. Developer Convenience > > There has been a long-running discussion around making DTS submissions for > DPDK patches that introduce new features mandatory. We feel that making DTS > look more like a standard python project would help ease the transition > period. In addition, as mentioned before, IDE features like autocomplete > and jump-to-definition would be functional, and we feel that those are very > useful for learning a codebase. This will also enable any future changes to > be made with the aid of automated refactoring tools, reducing the risk of > breaking changes. > > 2. Automated Tooling > > Most automated tooling also cannot process the additions to the python > path. Some tools, like formatters, are file-oriented and work perfectly > fine right now, but others, such as flake8, do not. Due to python's nature > as a duck typed language, not knowing the base type of something (because > it can't be imported inside of the tool), means that everything it > interacts with also has an unknown type. This results in some tooling > choosing to treat it as always the correct type, and some tooling to treat > it as though it could be any type in the project. Both of these approaches > reduce the accuracy of the tooling. > > > What will change: > > * The main file, currently located at dts/framework/main.py, will be moved > to dts/main.py. > * __init__.py files will be added to all modules to allow for exporting > the types the modules needs to expose. > * Most import paths will be updated, it is expected that most imports will > change from "import logging" to "import framework.logging as logging", to > avoid needing to change any source code during the reorganization. > > What we need from the community: > > What the DTS working group needs from the community is to let us know if > there are other changes that should be made, or if this change would cause > issues for you. We anticipate that there will not be many issues with this > reorganization unless there is external tooling attached to DTS that is > incompatible with a standard python module format. > > Owen Hilyard > --00000000000002aa1905c94928f3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello everyone,

Since there was no = feedback for a week, the DTS working group has decided to move ahead with t= his proposal as written.

Owen Hilyard
<= br>
On Thu,= Aug 5, 2021 at 2:49 PM Owen Hilyard <ohilyard@iol.unh.edu> wrote:
Hello everyone,

The DTS working group has decided that, conditional upon no issues being= discovered, DTS should be re-organized into a more standard python project= =C2=A0format. We have a few reasons for this.

How = DTS currently is structured is only possible due to modifications of the py= thon interpreter search path. The specific modifications are adding all of = the directories with python code in them, such as framework, nic, and tests= , to the search path. This enables an import statement like "import lo= gging" to work in a test suite, as opposed to the "import framewo= rk.logging as logging" that would otherwise be needed. However, this c= hange is not detected by most python tooling. As a result, features like au= tocompletion and jump-to-definition=C2=A0are broken in many IDEs, and stati= c analysis tools can only do surface-level inspections, which also produce = a lot of false positives due to undefined=C2=A0variables. There are two cor= e reasons the=C2=A0DTS working group has decided to move in this direction.=

1. Developer Convenience=C2=A0

There has been a long-running discussion around making DTS submissio= ns for DPDK patches that introduce new features mandatory. We feel that mak= ing DTS look more like a standard python project would help ease the transi= tion period. In addition, as mentioned before, IDE features like autocomple= te and jump-to-definition would be functional, and we feel that those are v= ery useful for learning a codebase. This will also enable any future change= s to be made with the aid of automated refactoring tools, reducing the risk= of breaking changes.

2. Automated Tooling

Most automated tooling also cannot process the additions = to the python path. Some tools, like formatters, are file-oriented and work= perfectly fine right now, but others, such as flake8, do not. Due to pytho= n's nature as a duck typed language, not knowing the base type of somet= hing (because it can't be imported inside of the tool), means that ever= ything it interacts with also has an unknown type. This results in some too= ling choosing to treat it as always the correct type, and some tooling to t= reat it as though it could be any type in the project. Both of these approa= ches reduce the accuracy of the tooling.=C2=A0

What will change:

* The main file, currently located at = dts/framework/main.py, will be moved to dts/main.py.=C2=A0
* __in= it__.py files will be added to all modules to allow for exporting the types= the modules needs to expose.
* Most import paths will be updated= , it is expected that most imports will change from "import logging&qu= ot; to "import framework.logging as logging", to avoid needing to= change any source code during the reorganization.=C2=A0

What we need from the community:

What the D= TS working group needs from the community is to let us know if there are ot= her changes that should be made, or if this change would cause issues for y= ou. We anticipate that there will not be many issues with this reorganizati= on unless there is external tooling attached to DTS that is incompatible wi= th a standard python module format.=C2=A0

Owen Hil= yard
--00000000000002aa1905c94928f3--