From: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
To: dev@dpdk.org
Cc: Neil Horman <nhorman@tuxdriver.com>,
Bruce Richardson <bruce.richardson@intel.com>,
Thomas Monjalon <thomas@monjalon.net>,
robin.jarry@6wind.com, Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Subject: [dpdk-dev] [RFC PATCH 0/2] pmdinfogen: rewrite in Python
Date: Mon, 22 Jun 2020 03:45:01 +0300 [thread overview]
Message-ID: <20200622004503.29036-1-dmitry.kozliuk@gmail.com> (raw)
This is a PoC rewrite of pmdinfogen in Python with missing bits
described below and in commits. Community input is desired.
Pros:
1. Simpler build process without host apps.
2. Less build requirements (host libelf).
3. Easier debugging and maintenance with a high-level language.
4. Easier porting on Windows (only add new object format).
Cons:
1. No standard ELF or COFF module for Python
(amount of Python code without libelf on par with C code using it).
2. struct rte_pci_id must be synchronized with header file
(it's a few lines that never change).
There are no built-in or widely used Python libraries for ELF or COFF.
Some ELF-parsing libraries exist on PyPI, but they're not very handy for
the task and their installation would complicate build requirements.
Thus, elf.py implements its own parsing.
COFF support is underway, it's just not included in this RFC.
Amount of code is similar to elf.py.
Build is only tested on Linux x64_64.
If the community deems this RFC worth finishing, there are a few opens:
1. Support for >65K section headers seems present in current pmdinfogen.
However, the data it reads is not used after. Is it really needed?
2. How much error-handling is required? This is a build-time tool,
and Python gives nice stacktraces. However, segfaults are possible
in Python version due to pointer handling. IMO, error checking
must be just sufficient to prevent silent segfaults.
3. On Unix, pmdinfogen is called for each object file extracted with ar
from an .a library by a shell script. On Windows, other tools
have to be used, shell script will not work. On the other hand, COFF
library format is quite simple. Would it be appropriate for
pmdinfogen to handle it to avoid intermediate script?
---
Dmitry Kozlyuk (2):
pmdinfogen: prototype in Python
build: use Python pmdinfogen
buildtools/elf.py | 194 +++++++++++++++++++++++++++++++++++++++
buildtools/meson.build | 3 +-
buildtools/pmdinfogen.py | 144 +++++++++++++++++++++++++++++
drivers/meson.build | 2 +-
4 files changed, 340 insertions(+), 3 deletions(-)
create mode 100644 buildtools/elf.py
create mode 100755 buildtools/pmdinfogen.py
--
2.25.4
next reply other threads:[~2020-06-22 0:45 UTC|newest]
Thread overview: 88+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-06-22 0:45 Dmitry Kozlyuk [this message]
2020-06-22 0:45 ` [dpdk-dev] [RFC PATCH 1/2] pmdinfogen: prototype " Dmitry Kozlyuk
2020-06-22 0:45 ` [dpdk-dev] [RFC PATCH 2/2] build: use Python pmdinfogen Dmitry Kozlyuk
2020-06-22 12:41 ` [dpdk-dev] [RFC PATCH 0/2] pmdinfogen: rewrite in Python Neil Horman
2020-06-22 19:39 ` Dmitry Kozlyuk
2020-06-23 11:28 ` Neil Horman
2020-06-23 11:59 ` Bruce Richardson
2020-07-02 0:07 ` Dmitry Kozlyuk
2020-07-02 0:02 ` [dpdk-dev] [RFC PATCH v2 " Dmitry Kozlyuk
2020-07-02 0:02 ` [dpdk-dev] [RFC PATCH v2 1/3] pmdinfogen: prototype " Dmitry Kozlyuk
2020-07-02 0:02 ` [dpdk-dev] [RFC PATCH v2 2/3] build: use Python pmdinfogen Dmitry Kozlyuk
2020-07-02 0:02 ` [dpdk-dev] [RFC PATCH v2 3/3] doc/linux_gsg: require pyelftools for pmdinfogen Dmitry Kozlyuk
2020-07-06 12:52 ` Neil Horman
2020-07-06 13:24 ` Dmitry Kozlyuk
2020-07-06 16:46 ` Neil Horman
2020-07-08 0:53 ` [dpdk-dev] [PATCH v3 0/4] pmdinfogen: rewrite in Python Dmitry Kozlyuk
2020-07-08 0:53 ` [dpdk-dev] [PATCH v3 1/4] pmdinfogen: add Python implementation Dmitry Kozlyuk
2020-07-08 0:53 ` [dpdk-dev] [PATCH v3 2/4] build: use Python pmdinfogen Dmitry Kozlyuk
2020-07-08 0:53 ` [dpdk-dev] [PATCH v3 3/4] doc/linux_gsg: require pyelftools for pmdinfogen Dmitry Kozlyuk
2020-07-08 0:53 ` [dpdk-dev] [PATCH v3 4/4] pmdinfogen: remove C implementation Dmitry Kozlyuk
2020-07-08 21:23 ` [dpdk-dev] [PATCH v4 0/4] pmdinfogen: rewrite in Python Dmitry Kozlyuk
2020-07-08 21:23 ` [dpdk-dev] [PATCH v4 1/4] pmdinfogen: add Python implementation Dmitry Kozlyuk
2020-07-08 21:23 ` [dpdk-dev] [PATCH v4 2/4] build: use Python pmdinfogen Dmitry Kozlyuk
2020-07-21 14:04 ` Bruce Richardson
2020-07-21 14:59 ` Dmitry Kozlyuk
2020-07-08 21:23 ` [dpdk-dev] [PATCH v4 3/4] doc/linux_gsg: require pyelftools for pmdinfogen Dmitry Kozlyuk
2020-07-21 13:39 ` Bruce Richardson
2020-07-21 14:05 ` Bruce Richardson
2020-07-21 14:04 ` Bruce Richardson
2020-07-08 21:23 ` [dpdk-dev] [PATCH v4 4/4] pmdinfogen: remove C implementation Dmitry Kozlyuk
2020-07-09 10:42 ` [dpdk-dev] [PATCH v4 0/4] pmdinfogen: rewrite in Python Neil Horman
2020-07-21 13:51 ` Bruce Richardson
2020-09-27 21:47 ` [dpdk-dev] [PATCH v5 0/3] " Dmitry Kozlyuk
2020-09-27 21:47 ` [dpdk-dev] [PATCH v5 1/3] pmdinfogen: add Python implementation Dmitry Kozlyuk
2020-09-27 22:05 ` Stephen Hemminger
2020-09-27 21:47 ` [dpdk-dev] [PATCH v5 2/3] build: use Python pmdinfogen Dmitry Kozlyuk
2020-09-27 21:47 ` [dpdk-dev] [PATCH v5 3/3] pmdinfogen: remove C implementation Dmitry Kozlyuk
2020-09-27 23:15 ` Thomas Monjalon
2020-09-28 9:35 ` [dpdk-dev] [PATCH v5 0/3] pmdinfogen: rewrite in Python David Marchand
2020-10-04 1:59 ` [dpdk-dev] [PATCH v6 " Dmitry Kozlyuk
2020-10-04 1:59 ` [dpdk-dev] [PATCH v6 1/3] pmdinfogen: add Python implementation Dmitry Kozlyuk
2020-10-04 1:59 ` [dpdk-dev] [PATCH v6 2/3] build: use Python pmdinfogen Dmitry Kozlyuk
2020-10-04 1:59 ` [dpdk-dev] [PATCH v6 3/3] pmdinfogen: remove C implementation Dmitry Kozlyuk
2020-10-14 14:37 ` [dpdk-dev] [PATCH v6 0/3] pmdinfogen: rewrite in Python Maxime Coquelin
2020-10-14 15:40 ` Dmitry Kozlyuk
2020-10-14 18:31 ` [dpdk-dev] [PATCH v7 " Dmitry Kozlyuk
2020-10-14 18:31 ` [dpdk-dev] [PATCH v7 1/3] pmdinfogen: add Python implementation Dmitry Kozlyuk
2020-10-14 18:31 ` [dpdk-dev] [PATCH v7 2/3] build: use Python pmdinfogen Dmitry Kozlyuk
2020-10-14 18:31 ` [dpdk-dev] [PATCH v7 3/3] pmdinfogen: remove C implementation Dmitry Kozlyuk
2020-10-20 16:02 ` [dpdk-dev] [PATCH v7 0/3] pmdinfogen: rewrite in Python David Marchand
2020-10-20 17:45 ` Dmitry Kozlyuk
2020-10-20 22:09 ` Dmitry Kozlyuk
2020-10-20 17:44 ` [dpdk-dev] [PATCH v8 " Dmitry Kozlyuk
2020-10-20 17:44 ` [dpdk-dev] [PATCH v8 1/3] pmdinfogen: add Python implementation Dmitry Kozlyuk
2020-10-20 17:44 ` [dpdk-dev] [PATCH v8 2/3] build: use Python pmdinfogen Dmitry Kozlyuk
2020-10-21 9:00 ` Bruce Richardson
2021-01-20 0:05 ` Thomas Monjalon
2021-01-20 7:23 ` Dmitry Kozlyuk
2021-01-20 10:24 ` Thomas Monjalon
2021-01-22 20:31 ` Dmitry Kozlyuk
2021-01-22 20:57 ` Thomas Monjalon
2021-01-22 22:24 ` Dmitry Kozlyuk
2021-01-23 11:38 ` Thomas Monjalon
2021-01-24 20:52 ` Dmitry Kozlyuk
2021-01-25 9:25 ` Kinsella, Ray
2021-01-25 10:01 ` Kinsella, Ray
2021-01-25 10:29 ` David Marchand
2021-01-25 10:46 ` Kinsella, Ray
2021-01-25 11:03 ` Thomas Monjalon
2021-01-25 10:05 ` Dmitry Kozlyuk
2021-01-25 10:11 ` Kinsella, Ray
2021-01-25 10:31 ` Dmitry Kozlyuk
2020-10-20 17:44 ` [dpdk-dev] [PATCH v8 3/3] pmdinfogen: remove C implementation Dmitry Kozlyuk
2020-10-26 16:46 ` Jie Zhou
2021-01-22 22:43 ` [dpdk-dev] [PATCH v9 0/3] pmdinfogen: rewrite in Python Dmitry Kozlyuk
2021-01-22 22:43 ` [dpdk-dev] [PATCH v9 1/3] pmdinfogen: add Python implementation Dmitry Kozlyuk
2021-01-22 22:43 ` [dpdk-dev] [PATCH v9 2/3] build: use Python pmdinfogen Dmitry Kozlyuk
2021-01-22 22:43 ` [dpdk-dev] [PATCH v9 3/3] pmdinfogen: remove C implementation Dmitry Kozlyuk
2021-01-24 20:51 ` [dpdk-dev] [PATCH v10 0/3] pmdinfogen: rewrite in Python Dmitry Kozlyuk
2021-01-24 20:51 ` [dpdk-dev] [PATCH v10 1/3] pmdinfogen: add Python implementation Dmitry Kozlyuk
2021-01-24 20:51 ` [dpdk-dev] [PATCH v10 2/3] build: use Python pmdinfogen Dmitry Kozlyuk
2021-01-25 10:12 ` Thomas Monjalon
2021-01-24 20:51 ` [dpdk-dev] [PATCH v10 3/3] pmdinfogen: remove C implementation Dmitry Kozlyuk
2021-01-25 13:13 ` [dpdk-dev] [PATCH v10 0/3] pmdinfogen: rewrite in Python Thomas Monjalon
2021-01-25 16:08 ` Brandon Lo
2021-02-02 8:48 ` Tal Shnaiderman
2021-01-25 18:51 ` Ali Alnubani
2021-01-25 22:15 ` Dmitry Kozlyuk
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200622004503.29036-1-dmitry.kozliuk@gmail.com \
--to=dmitry.kozliuk@gmail.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=nhorman@tuxdriver.com \
--cc=robin.jarry@6wind.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).