From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4DC59A00C3; Tue, 20 Sep 2022 19:48:36 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E890940E0F; Tue, 20 Sep 2022 19:48:35 +0200 (CEST) Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2049.outbound.protection.outlook.com [40.107.95.49]) by mails.dpdk.org (Postfix) with ESMTP id 919494069B for ; Tue, 20 Sep 2022 19:48:33 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LklPPchojEJn52wF5A15GoN3Uw6Uxq+3t9jMmBQlu+AOkp4c+TDZTB4o5MvHaJesT8qF1DDQJ3O6Z43qHBLqhqmc1G1JzTsEuO+E/GVhBiiAA0hQbIorwkHNk4UwGEg0/LolrSXyYY8euQXRVMGjOzwpEMHcgQDOsP1lWCcI6OLpJH2+ubKzfZAfzQOUAKDxHxfeUIp0tm4vlbABA67o+z0ZoLz8UxPbeOP2DMv7G57Tu0SPKE55Y5pHQeFSaf1i3yFPatB9XA+4H5lrx/1zo/P4IbVB8A0fRS8V/9vQltTSsxlDsT2p4kM9GRIteQnRHmM9tOf/pgDFfe4eL3+PLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EJxmPg/FJBBx+Sm2mA6yyQbGEyFWIlgHcdOhu3CGVCY=; b=KKad5V8a3qLGQJg0cpJly1M0HWWYbkbH1OlsdnCE3i9xAqQgkN1LRARByH0OvfTi0KShwi80v6p/5alvv9G0FwJfBQz9AgUlEa4pLU6cmPF2fIzf7PObGD1UZ30NpiTUXOcA7IZfJByw0OAz0R/lTXMDMWICqCSKdDDI218FOlmiSFqGe9Cgzll4NquQ+sWfAn87Uh6TdxqiS3jsOoYuYkbLZvGdriDlpZwlVU04Q0ONQBajc4I/4p2yK4BUtUrivofKXXMOORWPmmy30ik/x4nmI7YXhNea+z48GRhKFliNzgo/0MBP7FJ+19syV2ZgRfHmhL1pKvzjDRs9B2sA/A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.80.198) smtp.rcpttodomain=redhat.com smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EJxmPg/FJBBx+Sm2mA6yyQbGEyFWIlgHcdOhu3CGVCY=; b=UBHR18MFnKGBDDkbhDWirZ/hUPRhR47jJiJyBtpXgmx/Njz9AuDEhcCg3YcMHWTkLUmJ7Wm4vWofmcnZCmm2b31b/6ObHN4ma4dMLVnMUMZxjIphdB8gvkBDrd2wz5ik6whAF/EKLehKe4A0+Su/obHfkLvSPn6PIHiMwR7Q4pI= Received: from SA0PR11CA0088.namprd11.prod.outlook.com (2603:10b6:806:d2::33) by SA2PR02MB7706.namprd02.prod.outlook.com (2603:10b6:806:14c::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5632.21; Tue, 20 Sep 2022 17:48:29 +0000 Received: from SN1NAM02FT0018.eop-nam02.prod.protection.outlook.com (2603:10b6:806:d2:cafe::22) by SA0PR11CA0088.outlook.office365.com (2603:10b6:806:d2::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5654.14 via Frontend Transport; Tue, 20 Sep 2022 17:48:29 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.80.198) smtp.mailfrom=xilinx.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates 149.199.80.198 as permitted sender) receiver=protection.outlook.com; client-ip=149.199.80.198; helo=xir-pvapexch01.xlnx.xilinx.com; pr=C Received: from xir-pvapexch01.xlnx.xilinx.com (149.199.80.198) by SN1NAM02FT0018.mail.protection.outlook.com (10.97.5.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5632.12 via Frontend Transport; Tue, 20 Sep 2022 17:48:29 +0000 Received: from xir-pvapexch01.xlnx.xilinx.com (172.21.17.15) by xir-pvapexch01.xlnx.xilinx.com (172.21.17.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Tue, 20 Sep 2022 18:48:28 +0100 Received: from smtp.xilinx.com (172.21.105.198) by xir-pvapexch01.xlnx.xilinx.com (172.21.17.15) with Microsoft SMTP Server id 15.1.2375.24 via Frontend Transport; Tue, 20 Sep 2022 18:48:27 +0100 Envelope-to: rjarry@redhat.com, dev@dpdk.org, olivier.matz@6wind.com, bruce.richardson@intel.com Received: from [10.71.194.74] (port=58830) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1oahMB-0000Vk-Qt; Tue, 20 Sep 2022 18:48:27 +0100 Message-ID: Date: Tue, 20 Sep 2022 18:48:27 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.2.2 Subject: Re: [PATCH v4] usertools: rewrite pmdinfo Content-Language: en-US To: Robin Jarry , CC: Olivier Matz , Bruce Richardson References: <20220913105811.84384-1-robin@jarry.cc> <20220920104212.476889-1-rjarry@redhat.com> From: Ferruh Yigit In-Reply-To: <20220920104212.476889-1-rjarry@redhat.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1NAM02FT0018:EE_|SA2PR02MB7706:EE_ X-MS-Office365-Filtering-Correlation-Id: 3e0919ae-f027-4757-f484-08da9b3053d8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2uQTUYfcIDT6XGA+JTOv3gDlMEA5Z/p/AxDEzWs0aqhn9bcPJgKQunHJG7AhxCIUOJ3/zinYyhfYB92yPPt2Ji0OnMVvn7arpxj+vTJBNa0xXAzRvkArYQNTQiqIEyI9ulpRwxg1thwpOUbzqM4coCI3zo9LzUwWlwkqAj6JYzWAb4S+7p6PsWaQTunJPkEQ0Kn2dU67IR0M2yjDXswNbsRWe1xiCTETyaoVy2oPayps5HUAVZaCJCSQ7Er40/zCAJN7chDCr0zZ0JOUuB0Xfkhfhx84YuGejH2wkkWCDo66WXi27HkepDnXxUUCwOxJ986akjK8QcWbFWvLo1WnlPF9VtL9ME/xO4nUPTwVDz7Bl5bQdJunoum3bTl27zQS6U7AZr36S5Fmb7awtC3+e6yQZ/dS2Vjbc5v/zF424UBv5DoZYcr8IhiUY79JOH24MqPV+C7c35YUcZe4BgSa5sloPXolItjmcf4lMBc/HjvGlsmnuxzoxmFhgTzA3M1gFt2FtMVhjlEKC4lkceNzxXDPzIYSgifP0g8kKl5BdPAVvKgDHgxSiZZvbe46J0IxZaEN5x6et1Nnz4HEh/VoabFOsU2ZhUyrOwvt65wTJ9uqwRSKhoyI4oPTO+ae/s/GTk9I2J+y+SggU4JfKuNEm3qpkwOE3xSx4ff1A/00TzHO6y2AZQZxWVWIHHin5zwrwKIh5W3uLnotnO7RRPnYMPQEEVty53sH0hmvIJ2rZWyyHkQwAC8fdWrfBgtPu8blr0dk/j7fJGd8PeF6oAJ6cI21EcaHehQyrRDd6RS1YCQLgYiGKxlcmf75cEkCpb52rSM64HRIwrTOxYKRuNXcng== X-Forefront-Antispam-Report: CIP:149.199.80.198; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:xir-pvapexch01.xlnx.xilinx.com; PTR:unknown-80-198.xilinx.com; CAT:NONE; SFS:(13230022)(4636009)(136003)(376002)(39860400002)(396003)(346002)(451199015)(36840700001)(46966006)(40470700004)(356005)(426003)(82310400005)(336012)(8936002)(2616005)(9786002)(186003)(7636003)(41300700001)(54906003)(2906002)(40480700001)(26005)(478600001)(40460700003)(316002)(110136005)(31696002)(70586007)(36756003)(53546011)(8676002)(70206006)(4326008)(44832011)(5660300002)(36860700001)(83380400001)(47076005)(82740400003)(31686004)(50156003)(43740500002); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2022 17:48:29.1465 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3e0919ae-f027-4757-f484-08da9b3053d8 X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.80.198]; Helo=[xir-pvapexch01.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: SN1NAM02FT0018.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA2PR02MB7706 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 9/20/2022 11:42 AM, Robin Jarry wrote: > > dpdk-pmdinfo.py does not produce any parseable output. The -r/--raw flag > merely prints multiple independent JSON lines which cannot be fed > directly to any JSON parser. Moreover, the script complexity is rather > high for such a simple task: extracting PMD_INFO_STRING from .rodata ELF > sections. Rewrite it so that it can produce valid JSON. > > Remove the PCI database parsing for PCI-ID to Vendor-Device names > conversion. This should be done by external scripts (if really needed). > > Here are some examples of use with jq: > > Get the complete info for a given driver: > > ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \ > jq '.[] | select(.name == "dmadev_idxd_pci")' > { > "name": "dmadev_idxd_pci", > "params": "max_queues=0", > "kmod": "vfio-pci", > "pci_ids": [ > { > "vendor": "8086", > "device": "0b25" > } > ] > } > > Get only the required kernel modules for a given driver: > > ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \ > jq '.[] | select(.name == "net_i40e").kmod' > "* igb_uio | uio_pci_generic | vfio-pci" > > Get only the required kernel modules for a given device: > > ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \ > jq '.[] | select(.pci_ids[] | .vendor == "15b3" and .device == "1013").kmod' > "* ib_uverbs & mlx5_core & mlx5_ib" > > Print the list of drivers which define multiple parameters without > space separators: > > ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \ > jq '.[] | select(.params!=null and (.params|test("=[^ ]+="))) | {name, params}' > ... > > The script passes flake8, black, isort and pylint checks. > > I have tested this with a matrix of python/pyelftools versions: > > pyelftools > 0.22 0.23 0.24 0.25 0.26 0.27 0.28 0.29 > 3.6 ok ok ok ok ok ok ok ok > 3.7 ok ok ok ok ok ok ok ok > Python 3.8 ok ok ok ok ok ok ok ok > 3.9 ok ok ok ok ok ok ok ok > 3.10 fail fail fail fail ok ok ok ok > > All failures with python 3.10 are related to the same issue: > > File "elftools/construct/lib/container.py", line 5, in > from collections import MutableMapping > ImportError: cannot import name 'MutableMapping' from 'collections' > > Python 3.10 support is only available since pyelftools 0.26. The script > will only work with Python 3.6 and later. Update the minimal system > requirements and release notes. > > NB: The output produced by the legacy -r/--raw flag can be obtained with > the following command: > > strings build/app/dpdk-testpmd | sed -n 's/^PMD_INFO_STRING= //p' > > Cc: Olivier Matz > Cc: Ferruh Yigit > Cc: Bruce Richardson > Signed-off-by: Robin Jarry <...> > diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst > index 8c021cf0505e..67054f5acdc9 100644 > --- a/doc/guides/rel_notes/release_22_11.rst > +++ b/doc/guides/rel_notes/release_22_11.rst > @@ -84,6 +84,11 @@ API Changes > Also, make sure to start the actual text at the margin. > ======================================================= > > +* The ``dpdk-pmdinfo.py`` script was rewritten to produce valid JSON only. > + PCI-IDs parsing has been removed. > + To get a similar output to the (now removed) ``-r/--raw`` flag, you may use the following command:: > + > + strings $dpdk_binary_or_driver | sed -n 's/^PMD_INFO_STRING= //p' > Empty line is missing (in case there will be a new version for some other reason). Thanks for the update, Tested-by: Ferruh Yigit