From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
by inbox.dpdk.org (Postfix) with ESMTP id 534C6A053B;
Fri, 17 Jul 2020 15:50:55 +0200 (CEST)
Received: from [92.243.14.124] (localhost [127.0.0.1])
by dpdk.org (Postfix) with ESMTP id 375561C199;
Fri, 17 Jul 2020 15:50:03 +0200 (CEST)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com
(mail-eopbgr80087.outbound.protection.outlook.com [40.107.8.87])
by dpdk.org (Postfix) with ESMTP id 324A51C1A3
for <dev@dpdk.org>; Fri, 17 Jul 2020 15:50:00 +0200 (CEST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=cSV/4Uto9rSLF4wlxS5vzh0aySQGVtL0JfouFv7vIr8hLye/JgcSCW0u8+gXH0sWp/gobgVh+MI19H1DrmCC4I1HeogIK/VCgQqaiTrYxY132qoYoqf2kU2rxnTVDWt2qYADmIvUulAYroi4nZw6llLZcKSyqEi+sL08z7xvvRZY2LIh5MyE8tBgIzvB9NrGffw+3H/Chkf6Lw9YvhpYYgK51QMhIP9MI1MGcPGBlS6ze6+XKsiF2uKxkaSkO8XcvBVwhimF9ry6yUjKhyofJFIZLmBueee1EMKpkDx4zJ1zAIBeBRj58rhbWHy3aV/gLDZ6j2fuFvKdcpdc8dCpVA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=PnhS5755AppHts3f5KtzrFwaWPHpUTs6LoEVgRKB/QQ=;
b=UYWjhcdMFLHDSkHz2r5WsBXmMyiTtWNAXE9RzZbbj+15h/p61CGFmE37DtdLPE3lY1muNW7AEjxAhMz+RFoSTpdounUuJAIuaTojIBEvk8hFB3umZ5kyg5ZNhfr+LO5uGf6zVk7r7oYkLWFFl5T3pFVO2Xs5qDNDvdmLcj1dSaFsAbJ0nQwD6PEvknp6UxDKZJQlCoZz/Zeo3WauI8R4d1W6wjyPnIoNT3661ASSXb+Nh8TLYwmU1yj0VSur+njvqDEYI6iT19zHr6CiJRYowDxdeKfXCYVi7GPb8a8Vec2JABOV6yxBNU0F2tnmHW6KzDtdBJTzHRCWUIhXxAOpXw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com;
dkim=pass header.d=mellanox.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com;
s=selector1;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=PnhS5755AppHts3f5KtzrFwaWPHpUTs6LoEVgRKB/QQ=;
b=Tyy1YJw98wIesc376Nmb52wVUB8BdTQSMcPs5ZBFzrQrS5hMgjsQFhNvpXYcIcvj8Zd3fa1ARwIsBB9mbU1ZKzJkQfKkfwDAVeARv338R6wi0755PUi/QUrsrvwUVKHI+Z5oJCpDH+Fz6ERN8/hMr+nolEKiLh5ZmO2xR6XJSQo=
Authentication-Results: dpdk.org; dkim=none (message not signed)
header.d=none;dpdk.org; dmarc=none action=none header.from=mellanox.com;
Received: from AM0PR05MB4866.eurprd05.prod.outlook.com (2603:10a6:208:c0::32)
by AM0PR05MB4355.eurprd05.prod.outlook.com (2603:10a6:208:59::24)
with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3174.21; Fri, 17 Jul
2020 13:49:59 +0000
Received: from AM0PR05MB4866.eurprd05.prod.outlook.com
([fe80::d44d:a804:c730:d2b7]) by AM0PR05MB4866.eurprd05.prod.outlook.com
([fe80::d44d:a804:c730:d2b7%2]) with mapi id 15.20.3174.026; Fri, 17 Jul 2020
13:49:59 +0000
From: Parav Pandit <parav@mellanox.com>
To: dev@dpdk.org, grive@u256.net, ferruh.yigit@intel.com, thomas@monjalon.net
Cc: rasland@mellanox.com, orika@mellanox.com, matan@mellanox.com,
joyce.kong@arm.com, Parav Pandit <parav@mellanox.com>
Date: Fri, 17 Jul 2020 16:49:21 +0300
Message-Id: <20200717134924.922390-7-parav@mellanox.com>
X-Mailer: git-send-email 2.26.2
In-Reply-To: <20200717134924.922390-1-parav@mellanox.com>
References: <20200610171728.89-2-parav@mellanox.com>
<20200717134924.922390-1-parav@mellanox.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-ClientProxiedBy: DM3PR03CA0013.namprd03.prod.outlook.com
(2603:10b6:0:50::23) To AM0PR05MB4866.eurprd05.prod.outlook.com
(2603:10a6:208:c0::32)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from sw-mtx-036.mtx.labs.mlnx (208.176.44.194) by
DM3PR03CA0013.namprd03.prod.outlook.com (2603:10b6:0:50::23) with Microsoft
SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.20.3195.19 via Frontend Transport; Fri, 17 Jul 2020 13:49:57 +0000
X-Mailer: git-send-email 2.26.2
X-Originating-IP: [208.176.44.194]
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-HT: Tenant
X-MS-Office365-Filtering-Correlation-Id: 0387faec-1ecf-45b2-05c9-08d82a584ba1
X-MS-TrafficTypeDiagnostic: AM0PR05MB4355:
X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtFwd,ExtAddr
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <AM0PR05MB43556559FD283BE950262537D17C0@AM0PR05MB4355.eurprd05.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:7219;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: wsI1FTEP+EGMBrGMD1lDQ104unZXmVBpqMxo9elGl1UjDfo4MnBUeaEAgVKQDWkxrhdG1QmmlopOO+eAhHxgmYsdr5nrgcVlkSxX+YGliTC2O+VreQo3za52qet5YT5myuDXSJagJFBxk96z4AoAefj0tRpr4cdiZgWh1OOSuY9XD2YX0bZgNNTPX0rgnMm/cKIUKhjqSS7fTpn4H+e0T3AQW44+y25k98njTTrugfGDdS3onge5ke7Ay20zzYVbo4zzedoZDpadU85luMSk/VI1+xt1z0vWP1LcB3YrpKXryUytgTmNKOV7uYcQwAuM
X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
IPV:NLI; SFV:NSPM; H:AM0PR05MB4866.eurprd05.prod.outlook.com; PTR:; CAT:NONE;
SFTY:;
SFS:(4636009)(136003)(346002)(396003)(39860400002)(376002)(366004)(8936002)(6666004)(83380400001)(2616005)(107886003)(956004)(86362001)(478600001)(4326008)(16526019)(186003)(26005)(6512007)(52116002)(1076003)(6506007)(66476007)(66946007)(5660300002)(2906002)(8676002)(66556008)(316002)(36756003)(6486002);
DIR:OUT; SFP:1101;
X-MS-Exchange-AntiSpam-MessageData: fwWt9d/OnqfDqsOddzNI1EczSBxGXawbL1iF97pBuqEhsQ+PbdtEVUao0g97z87gMEyJ9ToFU2LgORobBgeESNrKA2nfSl5L5xTBV8hHXAPGYFdNdJpUpn+mRqWSF4BS/J9lQ8TQarcFYLQ57y0li8ntLBcD7KG2oCT5Kzl44h1mV7EqjkWGj09UyxuBDOw9SygocGWOK+YmIFvv1YledidqcrR70500q8/3T+PdVndVow8AWjisD4wH+VhEn3uAeELj+6KqTJ4LdCEUdGrtvaYS13w4Evbq3S9wUdDAx+SbY/0SMvPp0lhIuBW7Oeygqd3qn3oieL2UKzA5sV+Iq3ftVCyu9dcke1NRBbz0sLMwG1zgpRNfjzdeAALruiJ6AA2qNr39Gln997CfiroAwZ3cPBYW6mLJ2Or7u+xQIwV2UtEDUIcsI4jytZz08aqUmM5RShhzwCNiWV7hAvOoKGWFkffhtqLzgnSR5X90WWSW4yGQSJqc7s2IZ1aTn1pI
X-OriginatorOrg: Mellanox.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0387faec-1ecf-45b2-05c9-08d82a584ba1
X-MS-Exchange-CrossTenant-AuthSource: AM0PR05MB4866.eurprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jul 2020 13:49:59.1175 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: IMivGL99D9AxPCZcVA3Epo0jQfdjffkiSB7gqE3ByVGG0saQtx4ydJiEoz5gS+/9DONcv5dTWYAR/8Zcch+qTQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR05MB4355
Subject: [dpdk-dev] [PATCH v7 6/9] bus/mlx5_pci: add mlx5 PCI bus
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
Add mlx5 PCI bus which enables multiple mlx5 drivers to bind to single
pci device.
Signed-off-by: Parav Pandit <parav@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
---
Changelog:
v6->v7:
- Updated release notes
v5->v6:
- Updated Makefile for parallel shared build support
v4->v5:
- Merged maintainers update patch with this patch
v2->v3:
- Addressed comments from Thomas and Asaf
- Moved pci_driver structure instance as first in driver
- Removed white spaces at the end of line in diagram
- Address comments from Matan
- Removed CONFIG_RTE_LIBRTE_MLX5_PCI_BUS from config files
- Changed alignedment to mlx5 specific aligment instead of standard
DPDK
- Using uint32_t instead of mlx5_class enum
v1->v2:
- Address comments from Thomas and Gaetan
- Inheriting ret_pci_driver instead of rte_driver
- Added design and description of the mlx5_pci bus
---
MAINTAINERS | 5 ++
doc/guides/rel_notes/release_20_08.rst | 5 ++
drivers/bus/Makefile | 4 +
drivers/bus/meson.build | 2 +-
drivers/bus/mlx5_pci/Makefile | 39 +++++++++
drivers/bus/mlx5_pci/meson.build | 19 +++++
drivers/bus/mlx5_pci/mlx5_pci_bus.c | 14 ++++
drivers/bus/mlx5_pci/rte_bus_mlx5_pci.h | 84 +++++++++++++++++++
.../bus/mlx5_pci/rte_bus_mlx5_pci_version.map | 5 ++
9 files changed, 176 insertions(+), 1 deletion(-)
create mode 100644 drivers/bus/mlx5_pci/Makefile
create mode 100644 drivers/bus/mlx5_pci/meson.build
create mode 100644 drivers/bus/mlx5_pci/mlx5_pci_bus.c
create mode 100644 drivers/bus/mlx5_pci/rte_bus_mlx5_pci.h
create mode 100644 drivers/bus/mlx5_pci/rte_bus_mlx5_pci_version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index 3cd402b34..9e3bd1672 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -525,6 +525,11 @@ Intel FPGA bus
M: Rosen Xu <rosen.xu@intel.com>
F: drivers/bus/ifpga/
+Melllanox mlx5 PCI bus driver
+M: Parav Pandit <parav@mellaox.com>
+M: Matan Azrad <matan@mellanox.com>
+F: drivers/bus/mlx5_pci
+
NXP buses
M: Hemant Agrawal <hemant.agrawal@nxp.com>
M: Sachin Saxena <sachin.saxena@nxp.com>
diff --git a/doc/guides/rel_notes/release_20_08.rst b/doc/guides/rel_notes/release_20_08.rst
index 6f44ffdfc..917a5b8b3 100644
--- a/doc/guides/rel_notes/release_20_08.rst
+++ b/doc/guides/rel_notes/release_20_08.rst
@@ -226,6 +226,11 @@ New Features
See the :doc:`../sample_app_ug/l2_forward_real_virtual` for more
details of this parameter usage.
+* **Added Mellanox mlx5_pci bus driver.**
+
+ Added mlx5_pci bus driver that supports loading multiple PMDs for
+ Mellanox mlx5 devices. This bus PMD is auto selected when its
+ upper layer PMD need it.
Removed Items
-------------
diff --git a/drivers/bus/Makefile b/drivers/bus/Makefile
index cea3b55e6..a70f213c1 100644
--- a/drivers/bus/Makefile
+++ b/drivers/bus/Makefile
@@ -9,6 +9,10 @@ DIRS-$(CONFIG_RTE_LIBRTE_FSLMC_BUS) += fslmc
endif
DIRS-$(CONFIG_RTE_LIBRTE_IFPGA_BUS) += ifpga
DIRS-$(CONFIG_RTE_LIBRTE_PCI_BUS) += pci
+ifeq ($(findstring y,$(CONFIG_RTE_LIBRTE_MLX5_PMD)$(CONFIG_RTE_LIBRTE_MLX5_VDPA_PMD)),y)
+DEPDIRS-mlx5_pci := pci
+DIRS-y += mlx5_pci
+endif
DIRS-$(CONFIG_RTE_LIBRTE_VDEV_BUS) += vdev
DIRS-$(CONFIG_RTE_LIBRTE_VMBUS) += vmbus
diff --git a/drivers/bus/meson.build b/drivers/bus/meson.build
index 80de2d91d..b1381838d 100644
--- a/drivers/bus/meson.build
+++ b/drivers/bus/meson.build
@@ -1,7 +1,7 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2017 Intel Corporation
-drivers = ['dpaa', 'fslmc', 'ifpga', 'pci', 'vdev', 'vmbus']
+drivers = ['dpaa', 'fslmc', 'ifpga', 'pci', 'mlx5_pci', 'vdev', 'vmbus']
std_deps = ['eal']
config_flag_fmt = 'RTE_LIBRTE_@0@_BUS'
driver_name_fmt = 'rte_bus_@0@'
diff --git a/drivers/bus/mlx5_pci/Makefile b/drivers/bus/mlx5_pci/Makefile
new file mode 100644
index 000000000..dd24811a3
--- /dev/null
+++ b/drivers/bus/mlx5_pci/Makefile
@@ -0,0 +1,39 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2020 Mellanox Technologies, Ltd
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+#
+# library name
+#
+LIB = librte_bus_mlx5_pci.a
+
+CFLAGS += -O3 -Wall -Wextra
+CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -Wno-strict-prototypes
+CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5
+CFLAGS += -I$(BUILDDIR)/drivers/common/mlx5
+CFLAGS += -I$(RTE_SDK)/drivers/common/mlx5/linux
+CFLAGS += -I$(BUILDDIR)/drivers/bus/pci
+CFLAGS += -I$(BUILDDIR)/drivers/bus
+LDLIBS += -lrte_eal
+LDLIBS += -lrte_common_mlx5
+LDLIBS += -lrte_pci -lrte_bus_pci
+
+# versioning export map
+EXPORT_MAP := rte_bus_mlx5_pci_version.map
+
+SRCS-y += mlx5_pci_bus.c
+
+# DEBUG which is usually provided on the command-line may enable
+# CONFIG_RTE_LIBRTE_MLX5_DEBUG.
+ifeq ($(DEBUG),1)
+CONFIG_RTE_LIBRTE_MLX5_DEBUG := y
+endif
+
+#
+# Export include files
+#
+SYMLINK-y-include += rte_bus_mlx5_pci.h
+
+include $(RTE_SDK)/mk/rte.lib.mk
diff --git a/drivers/bus/mlx5_pci/meson.build b/drivers/bus/mlx5_pci/meson.build
new file mode 100644
index 000000000..64a17cbad
--- /dev/null
+++ b/drivers/bus/mlx5_pci/meson.build
@@ -0,0 +1,19 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2020 Mellanox Technologies Ltd
+
+deps += ['pci', 'bus_pci', 'common_mlx5']
+install_headers('rte_bus_mlx5_pci.h')
+sources = files('mlx5_pci_bus.c')
+
+cflags_options = [
+ '-std=c11',
+ '-Wno-strict-prototypes',
+ '-D_BSD_SOURCE',
+ '-D_DEFAULT_SOURCE',
+ '-D_XOPEN_SOURCE=600'
+]
+foreach option:cflags_options
+ if cc.has_argument(option)
+ cflags += option
+ endif
+endforeach
diff --git a/drivers/bus/mlx5_pci/mlx5_pci_bus.c b/drivers/bus/mlx5_pci/mlx5_pci_bus.c
new file mode 100644
index 000000000..66db3c7b0
--- /dev/null
+++ b/drivers/bus/mlx5_pci/mlx5_pci_bus.c
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include "rte_bus_mlx5_pci.h"
+
+static TAILQ_HEAD(mlx5_pci_bus_drv_head, rte_mlx5_pci_driver) drv_list =
+ TAILQ_HEAD_INITIALIZER(drv_list);
+
+void
+rte_mlx5_pci_driver_register(struct rte_mlx5_pci_driver *driver)
+{
+ TAILQ_INSERT_TAIL(&drv_list, driver, next);
+}
diff --git a/drivers/bus/mlx5_pci/rte_bus_mlx5_pci.h b/drivers/bus/mlx5_pci/rte_bus_mlx5_pci.h
new file mode 100644
index 000000000..9f8d22e2b
--- /dev/null
+++ b/drivers/bus/mlx5_pci/rte_bus_mlx5_pci.h
@@ -0,0 +1,84 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#ifndef _RTE_BUS_MLX5_PCI_H_
+#define _RTE_BUS_MLX5_PCI_H_
+
+/**
+ * @file
+ *
+ * RTE Mellanox PCI Bus Interface
+ * Mellanox ConnectX PCI device supports multiple class (net/vdpa/regex)
+ * devices. This bus enables creating such multiple class of devices on a
+ * single PCI device by allowing to bind multiple class specific device
+ * driver to attach to mlx5_pci bus driver.
+ *
+ * ----------- ------------ -----------------
+ * | mlx5 | | mlx5 | | mlx5 |
+ * | net pmd | | vdpa pmd | | new class pmd |
+ * ----------- ------------ -----------------
+ * \ | /
+ * \ | /
+ * \ ------------- /
+ * \______| mlx5 |_____ /
+ * | pci bus |
+ * -------------
+ * |
+ * -----------
+ * | mlx5 |
+ * | pci dev |
+ * -----------
+ *
+ * - mlx5 pci bus driver binds to mlx5 PCI devices defined by PCI
+ * ID table of all related mlx5 PCI devices.
+ * - mlx5 class driver such as net, vdpa, regex PMD defines its
+ * specific PCI ID table and mlx5 bus driver probes matching
+ * class drivers.
+ * - mlx5 pci bus driver is cental place that validates supported
+ * class combinations.
+ */
+
+#ifdef __cplusplus
+extern "C" {
+#endif /* __cplusplus */
+
+#include <rte_pci.h>
+#include <rte_bus_pci.h>
+
+#include <mlx5_common.h>
+
+/**
+ * A structure describing a mlx5 pci driver.
+ */
+struct rte_mlx5_pci_driver {
+ struct rte_pci_driver pci_driver; /**< Inherit core pci driver. */
+ uint32_t dev_class; /**< Class of this driver, enum mlx5_class */
+ TAILQ_ENTRY(rte_mlx5_pci_driver) next;
+};
+
+/**
+ * Register a mlx5_pci device driver.
+ *
+ * @param driver
+ * A pointer to a rte_mlx5_pci_driver structure describing the driver
+ * to be registered.
+ */
+__rte_internal
+void
+rte_mlx5_pci_driver_register(struct rte_mlx5_pci_driver *driver);
+
+#define RTE_PMD_REGISTER_MLX5_PCI(nm, drv) \
+ static const char *mlx5_pci_drvinit_fn_ ## nm; \
+ RTE_INIT(mlx5_pci_drvinit_fn_ ##drv) \
+ { \
+ (drv).driver.name = RTE_STR(nm); \
+ rte_mlx5_pci_driver_register(&drv); \
+ } \
+ RTE_PMD_EXPORT_NAME(nm, __COUNTER__)
+
+#ifdef __cplusplus
+}
+#endif /* __cplusplus */
+
+#endif /* _RTE_BUS_MLX5_PCI_H_ */
diff --git a/drivers/bus/mlx5_pci/rte_bus_mlx5_pci_version.map b/drivers/bus/mlx5_pci/rte_bus_mlx5_pci_version.map
new file mode 100644
index 000000000..4cfd3db10
--- /dev/null
+++ b/drivers/bus/mlx5_pci/rte_bus_mlx5_pci_version.map
@@ -0,0 +1,5 @@
+INTERNAL {
+ global:
+
+ rte_mlx5_pci_driver_register;
+};
--
2.26.2