From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 2D4754247C;
	Tue, 24 Jan 2023 15:08:26 +0100 (CET)
Received: from mails.dpdk.org (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id 320C242D47;
	Tue, 24 Jan 2023 15:08:04 +0100 (CET)
Received: from NAM10-DM6-obe.outbound.protection.outlook.com
 (mail-dm6nam10on2044.outbound.protection.outlook.com [40.107.93.44])
 by mails.dpdk.org (Postfix) with ESMTP id 3B8EA42D43
 for <dev@dpdk.org>; Tue, 24 Jan 2023 15:08:02 +0100 (CET)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=PWY7aJIXGXaMW4ksL0FLadZYogXGZ4YXuj9xhA9AMnOEV7lZcOHBc1xySE+bKM5ss+T6SdTwiGWbKGQTghNoge3OOT4GnzDmEkaQzrzSRvS2Nc/Vvi6zyTOt2NyUHXJ+c33/9pPyTkP3sNQXy7WcnYJR/xwVEFsbhRIRKmkouzxN4woHrW3gheBBmkd6gy5h0C2fjKPMwQTj9+woNHi1xRtMnRBkl4Tn3HG/agdPp+KoEOVGR6YNL/D7pDk8LRe6YnvFQFgrs9ZKpPNp7BxqRWZd9XE1GMnuZWbrKcywP8tsBHEhMxK8ZTAsSLFkTX2oO9emAZhw01zslxkB9hDqNA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; 
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
 bh=GiDzTkd0yyjAobMW/5JQtrlyuUeZjzAABq83NxC13Jo=;
 b=BCsxq24y4hNNOR5UvAfkJ7NFgyBT8tcCHtXwoqlNSVmnLtJICVd9788tLK1AQPf9SPMQ5Qg5K4GQH9taA/19jDuPpEAnSqUao+jZG4tK13IseOth7/8TDCEEkmQr8GFckVf02YX/oMSQN+tscpwkzBJ80ZkSYY2HeueaeHaUXdShiD4Fk5nw6pPTjPViC5WK96FA//QtechcCp1sbpyDdRtEu7K0QTVgWqRinVslCg3DnKO9Ych66cwwM7PBFhz1Ptkq0RoVY+Jt2qc1WGqRA1pCbyJiuL9EESPsdasZ1IlenN1eS75LKt5Blam/1KlZhaELqyRFlDgQCtp/VbvFaQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is
 165.204.84.17) smtp.rcpttodomain=dpdk.org smtp.mailfrom=amd.com; dmarc=pass
 (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com;
 dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; 
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=GiDzTkd0yyjAobMW/5JQtrlyuUeZjzAABq83NxC13Jo=;
 b=2//1c8xanf6qOEf0o7L5E2sikzwuZGgwdgt92sNMcAXJ8QC9QeBvaW7yOIhmSA2WxLeCQQWzDvYivVQCoPo1nN5U0DtkLZEl3BEBiUd0ttsOWiBcJe4dOlaAIXQUWTi01kKTxDVHinnR6qramqE+XBP7i4glw1FE/XQ6qTG9kW8=
Received: from BN0PR02CA0021.namprd02.prod.outlook.com (2603:10b6:408:e4::26)
 by IA1PR12MB8468.namprd12.prod.outlook.com (2603:10b6:208:445::16)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.33; Tue, 24 Jan
 2023 14:07:59 +0000
Received: from BN8NAM11FT101.eop-nam11.prod.protection.outlook.com
 (2603:10b6:408:e4:cafe::b5) by BN0PR02CA0021.outlook.office365.com
 (2603:10b6:408:e4::26) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.33 via Frontend
 Transport; Tue, 24 Jan 2023 14:07:59 +0000
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17)
 smtp.mailfrom=amd.com; dkim=none (message not signed)
 header.d=none;dmarc=pass action=none header.from=amd.com;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Received: from SATLEXMB04.amd.com (165.204.84.17) by
 BN8NAM11FT101.mail.protection.outlook.com (10.13.177.126) with Microsoft SMTP
 Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
 15.20.6002.13 via Frontend Transport; Tue, 24 Jan 2023 14:07:59 +0000
Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB04.amd.com
 (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 24 Jan
 2023 08:07:58 -0600
Received: from xhdipdslab49.xilinx.com (10.180.168.240) by SATLEXMB03.amd.com
 (10.181.40.144) with Microsoft SMTP Server id 15.1.2375.34 via
 Frontend Transport; Tue, 24 Jan 2023 08:07:56 -0600
From: Nipun Gupta <nipun.gupta@amd.com>
To: <dev@dpdk.org>, <thomas@monjalon.net>, <david.marchand@redhat.com>
CC: <ferruh.yigit@amd.com>, <harpreet.anand@amd.com>, <nikhil.agarwal@amd.com>
Subject: [RFC PATCH 4/6] bus/cdx: support plug unplug and dev iterator
Date: Tue, 24 Jan 2023 19:37:44 +0530
Message-ID: <20230124140746.594066-5-nipun.gupta@amd.com>
X-Mailer: git-send-email 2.25.1
In-Reply-To: <20230124140746.594066-1-nipun.gupta@amd.com>
References: <20230124140746.594066-1-nipun.gupta@amd.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: BN8NAM11FT101:EE_|IA1PR12MB8468:EE_
X-MS-Office365-Filtering-Correlation-Id: 0609f3ff-06f4-4e0c-f62f-08dafe146632
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: 0CfLfEmcmKtzo5FV2k2BwwzlnszCHL2uTh+Vu5ctfH49dKACvbR6RhMdUMafdz75t1BC06yPA3cdbTQKD6z0jcv4Ri+FMKCGXrsEwqeNPuLbHJLxEduL1uQdGkSTJrA17BSEnTmlbZhuj+6UGU+oLoxq6F6Eu34bRiDDa9Tua+OnvIM+cdqaEmMd3KtLdZCTO8/UurkcR2n2LBLdDDIjLu6KdV3L+KoV5InW6zmq513PBN4hb6IlnSYDDtLAS0L4ZohyPzdJpK10xOuchVfQxhEU3H1IkRj7IHX5Q+syYCuvasL3XSlwmX/cGN7BuAhBAb1E12EQRFUVa2IVKqu5Yc4Y8hVN1ayFzw7pvcBfd0lv0M23s4m00LqPrGWA/ymLjD0fmpdrtQ40WmlN1aHAy1W8M1Sg805Uz8BnD8rL07LSA9KE6iOurkS0tuTrsIWNINaPP2YguPXLD88nZ6UXPhdb0+E5gO+xY8Nf6QnI3NdPr9Y0PQR5kr/tgUqjmV23lt7QukE62cYU3M62QQyX9tgyjV5ZIpbM/0NZAGBig4fE8ML4jl01xGcvq5DmJ6kJ0lQxsSuhMwI+UpEFb7Z8oGe/x5g8MJ27XGHZfuDL8O4po36yciRDJ/iWxMUMLL3qpoKbdydiL8zwvNCo+hasih9hiUMZNriY/7OOhmo/SqMfc4yKZkQ3qyqHOLMT4B7rTqTwYD2LBu7kNV0teh9WV7vPsrSuxHaCawHio1Zz2cY=
X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:;
 IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE;
 SFS:(13230022)(4636009)(136003)(39860400002)(346002)(376002)(396003)(451199015)(40470700004)(36840700001)(46966006)(82740400003)(40460700003)(36860700001)(36756003)(81166007)(356005)(86362001)(8936002)(5660300002)(44832011)(2906002)(70586007)(41300700001)(8676002)(4326008)(70206006)(82310400005)(316002)(40480700001)(47076005)(426003)(2616005)(336012)(83380400001)(54906003)(110136005)(1076003)(478600001)(6666004)(26005)(186003)(36900700001);
 DIR:OUT; SFP:1101; 
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2023 14:07:59.3294 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 0609f3ff-06f4-4e0c-f62f-08dafe146632
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17];
 Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT101.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8468
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org

Add support for plugging and unplugging CDX devices on
the CDX bus. Also, CDX dev iterator support has been added
for the CDX bus.

Signed-off-by: Nipun Gupta <nipun.gupta@amd.com>
---
 drivers/bus/cdx/cdx.c         | 142 +++++++++++++++++++++++++++++++---
 drivers/bus/cdx/rte_bus_cdx.h |   1 +
 2 files changed, 133 insertions(+), 10 deletions(-)

diff --git a/drivers/bus/cdx/cdx.c b/drivers/bus/cdx/cdx.c
index d0adfb74ef..151d35ffd3 100644
--- a/drivers/bus/cdx/cdx.c
+++ b/drivers/bus/cdx/cdx.c
@@ -69,6 +69,7 @@
 #include <rte_eal_paging.h>
 #include <rte_errno.h>
 #include <rte_devargs.h>
+#include <rte_kvargs.h>
 #include <rte_malloc.h>
 #include <rte_memcpy.h>
 #include <rte_vfio.h>
@@ -81,6 +82,15 @@
 #define SYSFS_CDX_DEVICES "/sys/bus/cdx/devices"
 #define CDX_BUS_NAME	cdx
 
+enum cdx_params {
+	RTE_CDX_PARAM_NAME,
+};
+
+static const char * const cdx_params_keys[] = {
+	[RTE_CDX_PARAM_NAME] = "name",
+	NULL,
+};
+
 /**
  * @file
  * CDX probing using Linux sysfs.
@@ -88,7 +98,7 @@
 
 /* Add a device to CDX bus */
 static void
-rte_cdx_add_device(struct rte_cdx_device *cdx_dev)
+cdx_add_device(struct rte_cdx_device *cdx_dev)
 {
 	TAILQ_INSERT_TAIL(&rte_cdx_bus.device_list, cdx_dev, next);
 }
@@ -258,7 +268,7 @@ cdx_scan_one(const char *dirname, const char *dev_name)
 	}
 	dev->id.device_id = (uint16_t)tmp;
 
-	rte_cdx_add_device(dev);
+	cdx_add_device(dev);
 
 	return 0;
 
@@ -275,7 +285,7 @@ cdx_scan_one(const char *dirname, const char *dev_name)
  * list.
  */
 static int
-rte_cdx_scan(void)
+cdx_scan(void)
 {
 	struct dirent *e;
 	DIR *dir;
@@ -355,7 +365,7 @@ cdx_unmap_resource(void *requested_addr, size_t size)
  * Match the CDX Driver and Device using device id and vendor id.
  */
 static int
-rte_cdx_match(const struct rte_cdx_driver *cdx_drv,
+cdx_match(const struct rte_cdx_driver *cdx_drv,
 		const struct rte_cdx_device *cdx_dev)
 {
 	const struct rte_cdx_id *id_table;
@@ -381,7 +391,7 @@ rte_cdx_match(const struct rte_cdx_driver *cdx_drv,
  * driver.
  */
 static int
-rte_cdx_probe_one_driver(struct rte_cdx_driver *dr,
+cdx_probe_one_driver(struct rte_cdx_driver *dr,
 		struct rte_cdx_device *dev)
 {
 	const char *dev_name = dev->device.name;
@@ -392,7 +402,7 @@ rte_cdx_probe_one_driver(struct rte_cdx_driver *dr,
 		return -EINVAL;
 
 	/* The device is not blocked; Check if driver supports it */
-	if (!rte_cdx_match(dr, dev))
+	if (!cdx_match(dr, dev))
 		/* Match of device and driver failed */
 		return 1;
 
@@ -420,6 +430,7 @@ rte_cdx_probe_one_driver(struct rte_cdx_driver *dr,
 	} else {
 		dev->device.driver = &dr->driver;
 	}
+	dev->driver = dr;
 
 	return ret;
 
@@ -446,7 +457,7 @@ cdx_probe_all_drivers(struct rte_cdx_device *dev)
 		return -EINVAL;
 
 	FOREACH_DRIVER_ON_CDXBUS(dr) {
-		rc = rte_cdx_probe_one_driver(dr, dev);
+		rc = cdx_probe_one_driver(dr, dev);
 		if (rc < 0)
 			/* negative value is an error */
 			return rc;
@@ -560,6 +571,71 @@ cdx_find_device(const struct rte_device *start, rte_dev_cmp_t cmp,
 	return NULL;
 }
 
+/* Remove a device from CDX bus */
+static void
+cdx_remove_device(struct rte_cdx_device *cdx_dev)
+{
+	TAILQ_REMOVE(&rte_cdx_bus.device_list, cdx_dev, next);
+}
+
+/*
+ * If vendor/device ID match, call the remove() function of the
+ * driver.
+ */
+static int
+cdx_detach_dev(struct rte_cdx_device *dev)
+{
+	struct rte_cdx_driver *dr;
+	int ret = 0;
+
+	if (dev == NULL)
+		return -EINVAL;
+
+	dr = dev->driver;
+
+	CDX_BUS_DEBUG("detach device %s using driver: %s",
+		dev->device.name, dr->driver.name);
+
+	if (dr->remove) {
+		ret = dr->remove(dev);
+		if (ret < 0)
+			return ret;
+	}
+
+	/* clear driver structure */
+	dev->driver = NULL;
+	dev->device.driver = NULL;
+
+	rte_cdx_unmap_device(dev);
+
+	rte_intr_instance_free(dev->intr_handle);
+	dev->intr_handle = NULL;
+
+	return 0;
+}
+
+static int
+cdx_plug(struct rte_device *dev)
+{
+	return cdx_probe_all_drivers(RTE_DEV_TO_CDX_DEV(dev));
+}
+
+static int
+cdx_unplug(struct rte_device *dev)
+{
+	struct rte_cdx_device *cdx_dev;
+	int ret;
+
+	cdx_dev = RTE_DEV_TO_CDX_DEV(dev);
+	ret = cdx_detach_dev(cdx_dev);
+	if (ret == 0) {
+		cdx_remove_device(cdx_dev);
+		rte_devargs_remove(dev->devargs);
+		free(cdx_dev);
+	}
+	return ret;
+}
+
 static int
 cdx_dma_map(struct rte_device *dev, void *addr, uint64_t iova, size_t len)
 {
@@ -589,7 +665,7 @@ cdx_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova, size_t len)
 }
 
 static enum rte_iova_mode
-rte_cdx_get_iommu_class(void)
+cdx_get_iommu_class(void)
 {
 	if (TAILQ_EMPTY(&rte_cdx_bus.device_list))
 		return RTE_IOVA_DC;
@@ -597,15 +673,61 @@ rte_cdx_get_iommu_class(void)
 	return RTE_IOVA_VA;
 }
 
+static int
+cdx_dev_match(const struct rte_device *dev,
+		const void *_kvlist)
+{
+	const struct rte_kvargs *kvlist = _kvlist;
+	const char *key = cdx_params_keys[RTE_CDX_PARAM_NAME];
+	const char *name;
+
+	/* no kvlist arg, all devices match */
+	if (kvlist == NULL)
+		return 0;
+
+	/* if key is present in kvlist and does not match, filter device */
+	name = rte_kvargs_get(kvlist, key);
+	if (name != NULL && strcmp(name, dev->name))
+		return -1;
+
+	return 0;
+}
+
+static void *
+cdx_dev_iterate(const void *start,
+		const char *str,
+		const struct rte_dev_iterator *it __rte_unused)
+{
+	rte_bus_find_device_t find_device;
+	struct rte_kvargs *kvargs = NULL;
+	struct rte_device *dev;
+
+	if (str != NULL) {
+		kvargs = rte_kvargs_parse(str, cdx_params_keys);
+		if (kvargs == NULL) {
+			CDX_BUS_ERR("cannot parse argument list %s", str);
+			rte_errno = EINVAL;
+			return NULL;
+		}
+	}
+	find_device = rte_cdx_bus.bus.find_device;
+	dev = find_device(start, cdx_dev_match, kvargs);
+	rte_kvargs_free(kvargs);
+	return dev;
+}
+
 struct rte_cdx_bus rte_cdx_bus = {
 	.bus = {
-		.scan = rte_cdx_scan,
+		.scan = cdx_scan,
 		.probe = cdx_probe,
 		.find_device = cdx_find_device,
+		.plug = cdx_plug,
+		.unplug = cdx_unplug,
 		.parse = cdx_parse,
 		.dma_map = cdx_dma_map,
 		.dma_unmap = cdx_dma_unmap,
-		.get_iommu_class = rte_cdx_get_iommu_class,
+		.get_iommu_class = cdx_get_iommu_class,
+		.dev_iterate = cdx_dev_iterate,
 	},
 	.device_list = TAILQ_HEAD_INITIALIZER(rte_cdx_bus.device_list),
 	.driver_list = TAILQ_HEAD_INITIALIZER(rte_cdx_bus.driver_list),
diff --git a/drivers/bus/cdx/rte_bus_cdx.h b/drivers/bus/cdx/rte_bus_cdx.h
index 43d75bc3b4..564fa8d047 100644
--- a/drivers/bus/cdx/rte_bus_cdx.h
+++ b/drivers/bus/cdx/rte_bus_cdx.h
@@ -78,6 +78,7 @@ struct rte_cdx_id {
 struct rte_cdx_device {
 	RTE_TAILQ_ENTRY(rte_cdx_device) next;	/**< Next probed CDX device. */
 	struct rte_device device;		/**< Inherit core device */
+	struct rte_cdx_driver *driver;		/**< CDX driver used in probing */
 	struct rte_cdx_id id;			/**< CDX ID. */
 	struct rte_mem_resource mem_resource[CDX_MAX_RESOURCE];
 						/**< CDX Memory Resource */
-- 
2.25.1