From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 10EC4A04DB; Mon, 28 Sep 2020 18:47:42 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C6C021D90F; Mon, 28 Sep 2020 18:43:31 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 1A1571D8E1 for ; Mon, 28 Sep 2020 18:43:23 +0200 (CEST) IronPort-SDR: Xph3VT87Jdz8pW/l8c6H9vAmpVdUr2D1hiSemJR+uB/iN9NjG2ABrKRy6MlgqC/6RosAmIsh+q UAJ3qsBjUbbg== X-IronPort-AV: E=McAfee;i="6000,8403,9758"; a="223619840" X-IronPort-AV: E=Sophos;i="5.77,313,1596524400"; d="scan'208";a="223619840" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Sep 2020 09:43:23 -0700 IronPort-SDR: 8fkKa+gBMeQSOKOpK3vrEvVlh3R2MwGECRE/jcfMq/eRZOetIp0n2smHdf60G+El0EvZEgJGWx z6emgFuZE7xQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,313,1596524400"; d="scan'208";a="338250569" Received: from silpixa00399126.ir.intel.com ([10.237.222.4]) by fmsmga004.fm.intel.com with ESMTP; 28 Sep 2020 09:43:22 -0700 From: Bruce Richardson To: dev@dpdk.org Cc: patrick.fu@intel.com, Bruce Richardson , Kevin Laatz Date: Mon, 28 Sep 2020 17:42:33 +0100 Message-Id: <20200928164245.84997-14-bruce.richardson@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200928164245.84997-1-bruce.richardson@intel.com> References: <20200721095140.719297-1-bruce.richardson@intel.com> <20200928164245.84997-1-bruce.richardson@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v4 13/25] raw/ioat: include example configuration script X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Devices managed by the idxd kernel driver must be configured for DPDK use before it can be used by the ioat driver. This example script serves both as a quick way to get the driver set up with a simple configuration, and as the basis for users to modify it and create their own configuration scripts. Signed-off-by: Bruce Richardson Reviewed-by: Kevin Laatz --- doc/guides/rawdevs/ioat.rst | 2 + drivers/raw/ioat/dpdk_idxd_cfg.py | 79 +++++++++++++++++++++++++++++++ 2 files changed, 81 insertions(+) create mode 100755 drivers/raw/ioat/dpdk_idxd_cfg.py diff --git a/doc/guides/rawdevs/ioat.rst b/doc/guides/rawdevs/ioat.rst index 5b8d27980e..7c2a2d4570 100644 --- a/doc/guides/rawdevs/ioat.rst +++ b/doc/guides/rawdevs/ioat.rst @@ -50,6 +50,8 @@ The `accel-config `_ utility library can b .. note:: The device configuration can also be done by directly interacting with the sysfs nodes. + An example of how this may be done can be seen in the script ``dpdk_idxd_cfg.py`` + included in the driver source directory. There are some mandatory configuration steps before being able to use a device with an application. The internal engines, which do the copies or other operations, diff --git a/drivers/raw/ioat/dpdk_idxd_cfg.py b/drivers/raw/ioat/dpdk_idxd_cfg.py new file mode 100755 index 0000000000..bce4bb5bd4 --- /dev/null +++ b/drivers/raw/ioat/dpdk_idxd_cfg.py @@ -0,0 +1,79 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +""" +Configure an entire Intel DSA instance, using idxd kernel driver, for DPDK use +""" + +import sys +import argparse +import os +import os.path + + +class SysfsDir: + "Used to read/write paths in a sysfs directory" + def __init__(self, path): + self.path = path + + def read_int(self, filename): + "Return a value from sysfs file" + with open(os.path.join(self.path, filename)) as f: + return int(f.readline()) + + def write_values(self, values): + "write dictionary, where key is filename and value is value to write" + for filename, contents in values.items(): + with open(os.path.join(self.path, filename), "w") as f: + f.write(str(contents)) + + +def configure_dsa(dsa_id, queues): + "Configure the DSA instance with appropriate number of queues" + dsa_dir = SysfsDir(f"/sys/bus/dsa/devices/dsa{dsa_id}") + drv_dir = SysfsDir("/sys/bus/dsa/drivers/dsa") + + max_groups = dsa_dir.read_int("max_groups") + max_engines = dsa_dir.read_int("max_engines") + max_queues = dsa_dir.read_int("max_work_queues") + max_tokens = dsa_dir.read_int("max_tokens") + + # we want one engine per group + nb_groups = min(max_engines, max_groups) + for grp in range(nb_groups): + dsa_dir.write_values({f"engine{dsa_id}.{grp}/group_id": grp}) + + nb_queues = min(queues, max_queues) + if queues > nb_queues: + print(f"Setting number of queues to max supported value: {max_queues}") + + # configure each queue + for q in range(nb_queues): + wq_dir = SysfsDir(os.path.join(dsa_dir.path, f"wq{dsa_id}.{q}")) + wq_dir.write_values({"group_id": q % nb_groups, + "type": "user", + "mode": "dedicated", + "name": f"dpdk_wq{dsa_id}.{q}", + "priority": 1, + "size": int(max_tokens / nb_queues)}) + + # enable device and then queues + drv_dir.write_values({"bind": f"dsa{dsa_id}"}) + for q in range(nb_queues): + drv_dir.write_values({"bind": f"wq{dsa_id}.{q}"}) + + +def main(args): + "Main function, does arg parsing and calls config function" + arg_p = argparse.ArgumentParser( + description="Configure whole DSA device instance for DPDK use") + arg_p.add_argument('dsa_id', type=int, help="DSA instance number") + arg_p.add_argument('-q', metavar='queues', type=int, default=255, + help="Number of queues to set up") + parsed_args = arg_p.parse_args(args[1:]) + configure_dsa(parsed_args.dsa_id, parsed_args.q) + + +if __name__ == "__main__": + main(sys.argv) -- 2.25.1