From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3E5FAA0C41; Thu, 5 Aug 2021 20:49:54 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 11DA040143; Thu, 5 Aug 2021 20:49:54 +0200 (CEST) Received: from mail-oi1-f176.google.com (mail-oi1-f176.google.com [209.85.167.176]) by mails.dpdk.org (Postfix) with ESMTP id A3D4340040 for ; Thu, 5 Aug 2021 20:49:52 +0200 (CEST) Received: by mail-oi1-f176.google.com with SMTP id 26so8735971oiy.0 for ; Thu, 05 Aug 2021 11:49:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iol.unh.edu; s=unh-iol; h=mime-version:from:date:message-id:subject:to; bh=ysxej1Azxq78J+WylFk0dL2cSNfNXdTtYjIaUFyP3rU=; b=FTo5UOAFPXeGlHZC3WjyEFKhKV1vA2KbwZg/tIjdAbPc8do0CPVEUTRgu7hjQrJx0j yo0qrFPMk0WKfh57ddiiQud8LryEI8h+Nuig+EVmXwpa5McYvBHNWgYBg8yGbHyJ2Tez 0cCoPpn7riQaWAj5RQEw7Gcdvqkan46WeENkQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=ysxej1Azxq78J+WylFk0dL2cSNfNXdTtYjIaUFyP3rU=; b=qNkBqJ+muxNwDEdCqkg5NTALDeahV55tHBJLFQWgJLwZH6guweH3gPwj2XurXDrX6M rVfn7nm9SZxpFmwd5zvwhzSyeiuaiNQKVuv0WyCggvYAAbwnXzkHmGyHdIp9+qI/A6QT litEWziSq5x+4hIdUcJ/J71bwRb12Mk3rHimS07kI4+EhRqjCPcHbEScs/OF6JFS9bIA 594h3Fz7oeDKUU62fBcAyRDFyeR6ZayFjAf39WF329zv3Ebe5eaLk9msfLG5K2XsocrA liO6Rrz95+dX50DpaY9Ur+ImPA/KHe/I/igvhj8oIG8NBlADMSD/Tx0/pxZQ1kEPZStD MJiA== X-Gm-Message-State: AOAM533W6WYpRJzQwKXgZZaKfazK+20L67k1iF4IJ7lNtMRoGnzy6uhz yqdZoe4HdkwyCS1rA/C4tSii0/LVhvhrdGKBq9K/x07iY9MlOqOR X-Google-Smtp-Source: ABdhPJxIMzKmVisc4mgUgxsFSFCbRcDHZbYcdnRQvGLc5yccaQg6qbFY5UFgfxcZgvgtvOCNpKWz2saD4MC774+R/IY= X-Received: by 2002:a05:6808:1d1:: with SMTP id x17mr2044117oic.69.1628189391572; Thu, 05 Aug 2021 11:49:51 -0700 (PDT) MIME-Version: 1.0 From: Owen Hilyard Date: Thu, 5 Aug 2021 14:49:16 -0400 Message-ID: To: dts@dpdk.org Content-Type: multipart/alternative; boundary="0000000000004fe27c05c8d4629b" Subject: [dts] [RFC] Reorganization of DTS X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dts-bounces@dpdk.org Sender: "dts" --0000000000004fe27c05c8d4629b Content-Type: text/plain; charset="UTF-8" Hello everyone, The DTS working group has decided that, conditional upon no issues being discovered, DTS should be re-organized into a more standard python project format. We have a few reasons for this. How DTS currently is structured is only possible due to modifications of the python interpreter search path. The specific modifications are adding all of the directories with python code in them, such as framework, nic, and tests, to the search path. This enables an import statement like "import logging" to work in a test suite, as opposed to the "import framework.logging as logging" that would otherwise be needed. However, this change is not detected by most python tooling. As a result, features like autocompletion and jump-to-definition are broken in many IDEs, and static analysis tools can only do surface-level inspections, which also produce a lot of false positives due to undefined variables. There are two core reasons the DTS working group has decided to move in this direction. 1. Developer Convenience There has been a long-running discussion around making DTS submissions for DPDK patches that introduce new features mandatory. We feel that making DTS look more like a standard python project would help ease the transition period. In addition, as mentioned before, IDE features like autocomplete and jump-to-definition would be functional, and we feel that those are very useful for learning a codebase. This will also enable any future changes to be made with the aid of automated refactoring tools, reducing the risk of breaking changes. 2. Automated Tooling Most automated tooling also cannot process the additions to the python path. Some tools, like formatters, are file-oriented and work perfectly fine right now, but others, such as flake8, do not. Due to python's nature as a duck typed language, not knowing the base type of something (because it can't be imported inside of the tool), means that everything it interacts with also has an unknown type. This results in some tooling choosing to treat it as always the correct type, and some tooling to treat it as though it could be any type in the project. Both of these approaches reduce the accuracy of the tooling. What will change: * The main file, currently located at dts/framework/main.py, will be moved to dts/main.py. * __init__.py files will be added to all modules to allow for exporting the types the modules needs to expose. * Most import paths will be updated, it is expected that most imports will change from "import logging" to "import framework.logging as logging", to avoid needing to change any source code during the reorganization. What we need from the community: What the DTS working group needs from the community is to let us know if there are other changes that should be made, or if this change would cause issues for you. We anticipate that there will not be many issues with this reorganization unless there is external tooling attached to DTS that is incompatible with a standard python module format. Owen Hilyard --0000000000004fe27c05c8d4629b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello everyone,

The DTS working group h= as decided that, conditional upon no issues being discovered, DTS should be= re-organized into a more standard python project=C2=A0format. We have a fe= w reasons for this.

How DTS currently is structure= d is only possible due to modifications of the python interpreter search pa= th. The specific modifications are adding all of the directories with pytho= n code in them, such as framework, nic, and tests, to the search path. This= enables an import statement like "import logging" to work in a t= est suite, as opposed to the "import framework.logging as logging"= ; that would otherwise be needed. However, this change is not detected by m= ost python tooling. As a result, features like autocompletion and jump-to-d= efinition=C2=A0are broken in many IDEs, and static analysis tools can only = do surface-level inspections, which also produce a lot of false positives d= ue to undefined=C2=A0variables. There are two core reasons the=C2=A0DTS wor= king group has decided to move in this direction.

= 1. Developer Convenience=C2=A0

There has been a lo= ng-running discussion around making DTS submissions for DPDK patches that i= ntroduce new features mandatory. We feel that making DTS look more like a s= tandard python project would help ease the transition period. In addition, = as mentioned before, IDE features like autocomplete and jump-to-definition = would be functional, and we feel that those are very useful for learning a = codebase. This will also enable any future changes to be made with the aid = of automated refactoring tools, reducing the risk of breaking changes.

2. Automated Tooling

Most aut= omated tooling also cannot process the additions to the python path. Some t= ools, like formatters, are file-oriented and work perfectly fine right now,= but others, such as flake8, do not. Due to python's nature as a duck t= yped language, not knowing the base type of something (because it can't= be imported inside of the tool), means that everything it interacts with a= lso has an unknown type. This results in some tooling choosing to treat it = as always the correct type, and some tooling to treat it as though it could= be any type in the project. Both of these approaches reduce the accuracy o= f the tooling.=C2=A0


What will chan= ge:

* The main file, currently located at dts/framework/main.py, wil= l be moved to dts/main.py.=C2=A0
* __init__.py files will be adde= d to all modules to allow for exporting the types the modules needs to expo= se.
* Most import paths will be updated, it is expected that most= imports will change from "import logging" to "import framew= ork.logging as logging", to avoid needing to change any source code du= ring the reorganization.=C2=A0

What we need from t= he community:

What the DTS working group needs fro= m the community is to let us know if there are other changes that should be= made, or if this change would cause issues for you. We anticipate that the= re will not be many issues with this reorganization unless there is externa= l tooling attached to DTS that is incompatible with a standard python modul= e format.=C2=A0

Owen Hilyard
--0000000000004fe27c05c8d4629b--