From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2783AA0032; Tue, 13 Sep 2022 13:29:52 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C61E140151; Tue, 13 Sep 2022 13:29:51 +0200 (CEST) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2063.outbound.protection.outlook.com [40.107.92.63]) by mails.dpdk.org (Postfix) with ESMTP id 6E74E4021D for ; Tue, 13 Sep 2022 13:29:50 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Q0/cKiGxePx7uXc0lzkKOF7uKDHWe1SueTr8SIQszIQkyYHCTj9L4JZ40eex0wPJZcBbj8Wj6ToaOgE7hxIBCJTfhhZSVyUFvLMcrxCTflPvopw2+hfOtAhy/GrADarsSx8Uf4irjtjVNZaEumbfiRvyu55ms4yJkh5zH+tLpElmGbCskdDmscK7BKgeo9JExYCcB92HWhNqDoVqLMn4olj/5V0VQJIcXtOXgdit2bbmdKdIfVj88tO6RISTWjFYUV5m2jcPJUVOyq9Y1HmJOLkeHmjndp0q9GhpScqS8Lec8gQjAQ3Vs+4mz5zSFIrRVUD87OMA8qWbiAXNiXvg7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YzswM3adr65aCVHo8rh91x9eh9uKwp9nxFLJgLIndlg=; b=ep7T8vWMLmd8wRebWCKX1myi939UrwqAMrW+DF/oarTkGPz7+gDcgiqBRfHwI9ZdYJVlqo8KZlj8sEGPI0kAztlXOhVAx4TwTzAvMklDa5uroy2dzBJ6ZKdM2XF4/0xxZuc6jOpn7druw3vUfpgZ+dtIKEpGiD0+K0GdVJHfxMswNLA/ZxT+4LBotJ92VwErcOfxIpwnJ37/9p2OUARHQdoNnQ/d6XavtNt8Hx0EGNDL0Hvq8UtP+aH1GBkiBjH5Uezzhr/irt8sfubtKTaPWixU94vtjlwt2mCQE+Bx3RM58TyuTh4BNtHpoa71xrFFWJlm8GobYT8eAtaGLUXBwg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.80.198) smtp.rcpttodomain=jarry.cc smtp.mailfrom=xilinx.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=xilinx.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YzswM3adr65aCVHo8rh91x9eh9uKwp9nxFLJgLIndlg=; b=FKEBlSJt1GypJbo5M8HnlXtngDmkBJI6D+aRxD330d+Pe9pGkNgxHFV4hyVlL4Fxaltde2zkAj/a9HBFeNhPtUOSb5rOCWfCIHHpCauKFhcSa9UgFjlcS58qPsVkLBzm3b/MMly5VIBTFCJXdTP6bS2RPwOUZsxTeZOnPTvNeBY= Received: from BN9PR03CA0255.namprd03.prod.outlook.com (2603:10b6:408:ff::20) by BY5PR02MB7091.namprd02.prod.outlook.com (2603:10b6:a03:21c::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.12; Tue, 13 Sep 2022 11:29:48 +0000 Received: from BN1NAM02FT028.eop-nam02.prod.protection.outlook.com (2603:10b6:408:ff:cafe::5f) by BN9PR03CA0255.outlook.office365.com (2603:10b6:408:ff::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.19 via Frontend Transport; Tue, 13 Sep 2022 11:29:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.80.198) smtp.mailfrom=xilinx.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates 149.199.80.198 as permitted sender) receiver=protection.outlook.com; client-ip=149.199.80.198; helo=xir-pvapexch01.xlnx.xilinx.com; pr=C Received: from xir-pvapexch01.xlnx.xilinx.com (149.199.80.198) by BN1NAM02FT028.mail.protection.outlook.com (10.13.2.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5612.13 via Frontend Transport; Tue, 13 Sep 2022 11:29:47 +0000 Received: from xir-pvapexch02.xlnx.xilinx.com (172.21.17.17) by xir-pvapexch01.xlnx.xilinx.com (172.21.17.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Tue, 13 Sep 2022 12:29:34 +0100 Received: from smtp.xilinx.com (172.21.105.197) by xir-pvapexch02.xlnx.xilinx.com (172.21.17.17) with Microsoft SMTP Server id 15.1.2375.24 via Frontend Transport; Tue, 13 Sep 2022 12:29:34 +0100 Envelope-to: robin@jarry.cc, olivier.matz@6wind.com, dev@dpdk.org Received: from [10.71.194.74] (port=9528) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1oY46g-0002Jr-2Q; Tue, 13 Sep 2022 12:29:34 +0100 Message-ID: <467bb09c-34ec-811f-9781-bc7ad5d63e42@xilinx.com> Date: Tue, 13 Sep 2022 12:29:33 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.2.1 Subject: Re: [PATCH] usertools: rewrite pmdinfo Content-Language: en-US To: Robin Jarry CC: Olivier Matz , References: <20220913105811.84384-1-robin@jarry.cc> From: Ferruh Yigit In-Reply-To: <20220913105811.84384-1-robin@jarry.cc> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1NAM02FT028:EE_|BY5PR02MB7091:EE_ X-MS-Office365-Filtering-Correlation-Id: 1b450780-8cf7-4f11-d150-08da957b43cb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5U/sUMagVUIaXlw4Nw+nkSkI1r84jXz/RH7hRKHzWwfo1Qq2qotND/dRrnl9tO866YSkNq4FOg9ESWUdgLdN6fp8JiV/9JOHjjSiCphVaQxtQc5shaHcEL0+zgNq4T82XACkYNGseBKCVWDwT5UTLzY5v/w+RKTobeh0wm2XuETv7VXa8mgbjlfWYi4T6XkeuADIlwrX+FgYxK3Csusq1yqSBgvSvnHZhjIg1vtbgl0NNTh79wvwwHmZC/Z2nTGuxaAtIzRWm6fuPLju4NwWdeAJKyhtQx7tFTpqzEdlDsnFsPpofc5jRxwyFN10PzNmSBzQIBqP0JITe6pp3nO6yAjahWyNAZ+A3V0cOC3GZzVkEjKzXKl0EV/BGuma/uYdL9hmc+/cWaOIGqYOPzCwM4uom3selq92rbViVYT9FzzUs/N9bXSC7r3qZPWAT8j6Kay/8iT2ccphlT6+nE7koY8gcYpCGnsHRt3ek/M8IwXCbUOhrd8m/Yx5VDjGu/ddAP+VOvPMa567uISJgrJuBYAVONuR8b9PxoF7IkPrBSTgNtiuLrCYNSJ+qiC84Bq99Oy9A1FIlKW0H/TYBZFeeDMKtSKNpL5Aao7DYn2b8kgAua56EpaQBjuhYrybZQyAOnYftDVL/rE+qj7hEokdR32nRVehw2RO81YBWEtMaRo1adpyl0bn/p14/1YAiFrGiAzCbSdgDrI9RPCC5hAJcl1+pAcwsrhQ6Bg/h9dkaVmYPQufa+vrc2qZiGvIzQbRnIojVkB5zvJiLgM6OnmVVLSZ57H2NjbHTfu6THwpb5HK1trdq7Lah9TvvZBMWlKBKAV/oX5xLBN0HaTaBWrEAA== X-Forefront-Antispam-Report: CIP:149.199.80.198; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:xir-pvapexch01.xlnx.xilinx.com; PTR:unknown-80-198.xilinx.com; CAT:NONE; SFS:(13230022)(4636009)(396003)(376002)(39860400002)(346002)(136003)(451199015)(46966006)(36840700001)(40470700004)(316002)(31686004)(186003)(47076005)(82310400005)(41300700001)(26005)(336012)(82740400003)(70586007)(8936002)(40480700001)(9786002)(2906002)(70206006)(5660300002)(36756003)(8676002)(4326008)(478600001)(2616005)(356005)(40460700003)(31696002)(44832011)(83380400001)(53546011)(54906003)(7636003)(6916009)(426003)(36860700001)(50156003)(43740500002); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2022 11:29:47.5731 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1b450780-8cf7-4f11-d150-08da957b43cb X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.80.198]; Helo=[xir-pvapexch01.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: BN1NAM02FT028.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR02MB7091 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 9/13/2022 11:58 AM, Robin Jarry wrote: > dpdk-pmdinfo.py does not produce any parseable output. The -r/--raw flag > merely prints multiple independent JSON lines which cannot be fed > directly to any JSON parser. Moreover, the script complexity is rather > high for such a simple task: extracting PMD_INFO_STRING from .rodata ELF > sections. Rewrite it so that it can produce valid JSON. > > Remove the PCI database parsing for PCI-ID to Vendor-Device names > conversion. This should be done by external scripts (if really needed). > > Here are some examples of use with jq: > > Get the complete info for a given driver: > > ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \ > jq '.[] | select(.name == "dmadev_idxd_pci")' > { > "name": "dmadev_idxd_pci", > "params": "max_queues=0", > "kmod": "vfio-pci", > "devices": [ > { > "vendor_id": "8086", > "device_id": "0b25", > "subsystem_device_id": "ffff", > "subsystem_system_id": "ffff" > } > ] > } > > Get only the required kernel modules for a given driver: > > ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \ > jq '.[] | select(.name == "net_i40e").kmod' > "* igb_uio | uio_pci_generic | vfio-pci" > > Get only the required kernel modules for a given device: > > ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \ > jq '.[] | select(.devices[] | .vendor_id == "15b3" and .device_id == "1013").kmod' > "* ib_uverbs & mlx5_core & mlx5_ib" > > Print the list of drivers which define multiple parameters without > string separators: > > ~$ usertools/dpdk-pmdinfo.py build/app/dpdk-testpmd | \ > jq '.[] | select(.params!=null and (.params|test("=[^ ]+="))) | {name, params}' > ... > > The script passes flake8, black, isort and pylint checks. > > I have tested this with a matrix of python/pyelftools versions: > > pyelftools > 0.22 0.23 0.24 0.25 0.26 0.27 0.28 0.29 > 3.6 ok ok ok ok ok ok ok ok > 3.7 ok ok ok ok ok ok ok ok > Python 3.8 ok ok ok ok ok ok ok ok > 3.9 ok ok ok ok ok ok ok ok > 3.10 fail fail fail fail ok ok ok ok > > All failures with python 3.10 are related to the same issue: > > File "elftools/construct/lib/container.py", line 5, in > from collections import MutableMapping > ImportError: cannot import name 'MutableMapping' from 'collections' > > Python 3.10 support is only available since pyelftools 0.26. > > NB: The output produced by the legacy -r/--raw flag can be obtained with > the following command: > > strings build/app/dpdk-testpmd | sed -n 's/^PMD_INFO_STRING= //p' > > Cc: Olivier Matz > Signed-off-by: Robin Jarry Hi Robin, Thanks for the work. One of the major usecase of the script is to get information from binary drivers. So intentions of the script is to run it on drivers more than applications (dpdk-testpmd). When I run it on one of the .so drivers, it is generating some warnings [1], is this expected? [1] $ ./usertools/dpdk-pmdinfo.py ./build/drivers/librte_net_ixgbe.so warning: cannot find librte_ethdev.so.23 warning: cannot find librte_eal.so.23 warning: cannot find librte_kvargs.so.23 warning: cannot find librte_telemetry.so.23 warning: cannot find librte_net.so.23 warning: cannot find librte_mbuf.so.23 warning: cannot find librte_mempool.so.23 warning: cannot find librte_ring.so.23 warning: cannot find librte_meter.so.23 warning: cannot find librte_bus_pci.so.23 warning: cannot find librte_pci.so.23 warning: cannot find librte_bus_vdev.so.23 warning: cannot find librte_hash.so.23 warning: cannot find librte_rcu.so.23 warning: cannot find librte_security.so.23 warning: cannot find librte_cryptodev.so.23 [ { "name": "net_ixgbe_vf", "params": "pflink_fullchk=<0|1>", "kmod": "* igb_uio | vfio-pci", "devices": [ ... ...