From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 15636A0577; Tue, 14 Apr 2020 09:26:49 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 5E2151C126; Tue, 14 Apr 2020 09:26:47 +0200 (CEST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2087.outbound.protection.outlook.com [40.107.21.87]) by dpdk.org (Postfix) with ESMTP id DA71F1C125 for ; Tue, 14 Apr 2020 09:26:45 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Vi+yKLN9laEDORjWssdGtP9GzicmzC8gJMIrUYCjHFakoQzrivBfKTLH7yqERXGr8j89cmElvq1xVn/dR4mRjsCGtIMqj5ZOAVN9ov/6kw2HcyYOh36QWkMAkkKw/tbysjNfFaTBJaWCqNYPiouw3UYRHqatecVKOgHUA9PAgjgMLYi/Rp6jbLmy3/ojknLOMb6KJt7mitLKh9aQP554Dssv016uk89iHB7186Lehz+n+VKwroaQARoJ0eC0In9nSz8CjlFiyetUGMS2mQuxQ9Fp1vy+tiaiWT7x8WQlgSib+WCf3kDPw4bg8jH0icP/K3LYMeFJhOMjzX6tIKHZGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ev9ERutPPUA+ogS4cj4AyZwZeXPUOqeQWCfWtYOq8rs=; b=iB6bL6Velre5k9NJp8an3qYVuu7uMSIaA5/hyBbk3/ohY1WIMqYf0z1gZ0v9BCU+YK3Ce+Fjkj6umxDT8myeJ7uSfflP/69H923Iukc2kOVBwsg2GHgftxLzqIggxtMs7xRVh33355gV5ob4yx50QUa0CJvB52U4kiNHTsPfcNrVfcNC3YCR1yugoMWNfcw3CQJQTD+4s4sRBM3dYzWM6in5/GBSe7W3oPdzErz3jwZjUtJARqKEERKFlJs0EB9U27nDbz5ElcIH2xBnTkH3v0RM7b18cutrEbPuyD5YzT3SopzO1P7jWUaAGDV8JUlnzEJbX1//oHVzyAat5idHDw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ev9ERutPPUA+ogS4cj4AyZwZeXPUOqeQWCfWtYOq8rs=; b=NN6j6LRJSLCf0as3upZj1F/VKjHzQUn8MD1jgetac8kxSeABjFAjVt3xwusrXgM9X2bABUW3XUih1pUI+Blywg0PBx+jEZCgiK25I+uS5DDOdmJULUXjGIZhZz8w9Hv9O8V3FQnacjqrptWrYlCldYKa0TQmugbBTbP0us8AQX0= Received: from AM4PR05MB3265.eurprd05.prod.outlook.com (2603:10a6:205:8::26) by AM4PR05MB3489.eurprd05.prod.outlook.com (2603:10a6:205:11::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2900.15; Tue, 14 Apr 2020 07:26:41 +0000 Received: from AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::bdf0:88a3:3a39:4be4]) by AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::bdf0:88a3:3a39:4be4%7]) with mapi id 15.20.2900.028; Tue, 14 Apr 2020 07:26:41 +0000 From: Slava Ovsiienko To: Vu Pham , "dev@dpdk.org" CC: Ori Kam , Matan Azrad , Raslan Darawsheh , Vu Pham Thread-Topic: [PATCH v4 1/2] common/mlx5: refactor multi-process IPC handling codes to common driver Thread-Index: AQHWEdkDtV5XXYQ1Wk2Y9PsdgqXM/ah4N/gQ Date: Tue, 14 Apr 2020 07:26:41 +0000 Message-ID: References: <20200402192105.11606-1-vuhuoong@mellanox.com> <20200413211748.20500-1-vuhuong@mellanox.com> <20200413211748.20500-2-vuhuong@mellanox.com> In-Reply-To: <20200413211748.20500-2-vuhuong@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=viacheslavo@mellanox.com; x-originating-ip: [95.164.10.10] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 69f87f3e-9483-4071-8041-08d7e0452d5a x-ms-traffictypediagnostic: AM4PR05MB3489:|AM4PR05MB3489: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtFwd x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:660; x-forefront-prvs: 0373D94D15 x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM4PR05MB3265.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(346002)(366004)(39850400004)(376002)(136003)(396003)(8676002)(54906003)(81156014)(71200400001)(30864003)(316002)(110136005)(478600001)(33656002)(66946007)(186003)(107886003)(4326008)(2906002)(86362001)(55016002)(8936002)(5660300002)(53546011)(66446008)(66556008)(7696005)(52536014)(64756008)(6506007)(9686003)(26005)(76116006)(66476007)(559001)(579004); DIR:OUT; SFP:1101; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ZLSyDrbg9auD/no715+vfbnFZpCxYPuWgm1CvKLH3BGfO4nlj/jXzm3hN1uq2z5JEuYboBwOZU6uAXna4V+FONodd3QIBpnDTDvbJTsGUURItqUn0n6VgZ9wl8MTgJC1fvVzlfQnyDmKiyQ8g4o1GLtNBQuhz1lKFG13SN4agiMr9VMid87Fq6IDgVASGsJCehWbdmGEw/Aeh3SlHuy1vSIUiQf9p6Z7wuNj7TYc5wZD0r+2ochrhos1da88L8FsGwq1vFAnwyUpdsd3a/ubit92q5umX73YOwtSheGzrUZOnKKFv+M1nCRKTxTalHrYR8S+lNeVe7cMtiKjFunD8JUF5tA1AOdMQMo1M8TlnySclHLECzsCpYVIFfsfpYk4EBJkQqCBJHZn0Sb32g96v0HMdzgzEC1IF0ap8MGA2msROcTpHFY7oJL7N/N51O/P x-ms-exchange-antispam-messagedata: uBABF2jSc0dY2l4jQkUh92QQWLU4frNoY0Kz/E7DUCd8k+I35l48HsSQU0X9InUlUgeby6HuZ4sPVafLOZHT8qLc01Ha6eDYDONTpjUkQTtgpgZgOSUzQxUpNF3dgtE6Heax+fH0AbKg+vHxN9nj3Q== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 69f87f3e-9483-4071-8041-08d7e0452d5a X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Apr 2020 07:26:41.3536 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: IBjbSd5HdWTQxkv+A6ugQRo1IasWdBrqOEdxltPyXgeWi0l5J+f0OOCJVA58rksDHbJW4IO4C+6joU+NTtSeWdnOamslERDyUtF/91Na3O4= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR05MB3489 Subject: Re: [dpdk-dev] [PATCH v4 1/2] common/mlx5: refactor multi-process IPC handling codes to common driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Vu Pham > Sent: Tuesday, April 14, 2020 0:18 > To: dev@dpdk.org > Cc: Slava Ovsiienko ; Ori Kam > ; Matan Azrad ; Raslan > Darawsheh ; Vu Pham > Subject: [PATCH v4 1/2] common/mlx5: refactor multi-process IPC handling > codes to common driver >=20 > Refactor common multi-process handling codes from net PMD to common > driver. Using tuple mp_id{name, port_id} as standard input parameter for = all > multi-process IPC APIs instead of using rte_eth_dev. >=20 > Modify net PMD to use multi-process APIs from common driver. >=20 > Signed-off-by: Vu Pham Acked-by: Viacheslav Ovsiienko > --- > drivers/common/mlx5/Makefile | 3 +- > drivers/common/mlx5/meson.build | 1 + > drivers/common/mlx5/mlx5_common_mp.c | 188 > +++++++++++++++++++ > drivers/common/mlx5/mlx5_common_mp.h | 98 ++++++++++ > drivers/common/mlx5/rte_common_mlx5_version.map | 13 ++ > drivers/net/mlx5/mlx5.c | 15 +- > drivers/net/mlx5/mlx5.h | 43 +---- > drivers/net/mlx5/mlx5_mp.c | 234 ++----------------= ------ > drivers/net/mlx5/mlx5_mr.c | 2 +- > drivers/net/mlx5/mlx5_rxtx.c | 3 +- > 10 files changed, 336 insertions(+), 264 deletions(-) create mode 10064= 4 > drivers/common/mlx5/mlx5_common_mp.c > create mode 100644 drivers/common/mlx5/mlx5_common_mp.h >=20 > diff --git a/drivers/common/mlx5/Makefile b/drivers/common/mlx5/Makefile > index f32933d592..2a88492731 100644 > --- a/drivers/common/mlx5/Makefile > +++ b/drivers/common/mlx5/Makefile > @@ -17,6 +17,7 @@ endif > SRCS-y +=3D mlx5_devx_cmds.c > SRCS-y +=3D mlx5_common.c > SRCS-y +=3D mlx5_nl.c > +SRCS-y +=3D mlx5_common_mp.c > ifeq ($(CONFIG_RTE_IBVERBS_LINK_DLOPEN),y) > INSTALL-y-lib +=3D $(LIB_GLUE) > endif > @@ -46,7 +47,7 @@ endif > LDLIBS +=3D -lrte_eal -lrte_pci -lrte_kvargs -lrte_net >=20 > # A few warnings cannot be avoided in external headers. > -CFLAGS +=3D -Wno-error=3Dcast-qual -UPEDANTIC > +CFLAGS +=3D -Wno-error=3Dcast-qual -UPEDANTIC -DALLOW_EXPERIMENTAL_API >=20 > EXPORT_MAP :=3D rte_common_mlx5_version.map >=20 > diff --git a/drivers/common/mlx5/meson.build > b/drivers/common/mlx5/meson.build index f671710714..83671861c9 100644 > --- a/drivers/common/mlx5/meson.build > +++ b/drivers/common/mlx5/meson.build > @@ -55,6 +55,7 @@ sources =3D files( > 'mlx5_devx_cmds.c', > 'mlx5_common.c', > 'mlx5_nl.c', > + 'mlx5_common_mp.c', > ) > if not dlopen_ibverbs > sources +=3D files('mlx5_glue.c') > diff --git a/drivers/common/mlx5/mlx5_common_mp.c > b/drivers/common/mlx5/mlx5_common_mp.c > new file mode 100644 > index 0000000000..da55143bc1 > --- /dev/null > +++ b/drivers/common/mlx5/mlx5_common_mp.c > @@ -0,0 +1,188 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright 2019 6WIND S.A. > + * Copyright 2019 Mellanox Technologies, Ltd */ > + > +#include > +#include > + > +#include > +#include > + > +#include "mlx5_common_mp.h" > +#include "mlx5_common_utils.h" > + > +/** > + * Request Memory Region creation to the primary process. > + * > + * @param[in] mp_id > + * ID of the MP process. > + * @param addr > + * Target virtual address to register. > + * > + * @return > + * 0 on success, a negative errno value otherwise and rte_errno is set= . > + */ > +int > +mlx5_mp_req_mr_create(struct mlx5_mp_id *mp_id, uintptr_t addr) { > + struct rte_mp_msg mp_req; > + struct rte_mp_msg *mp_res; > + struct rte_mp_reply mp_rep; > + struct mlx5_mp_param *req =3D (struct mlx5_mp_param > *)mp_req.param; > + struct mlx5_mp_param *res; > + struct timespec ts =3D {.tv_sec =3D MLX5_MP_REQ_TIMEOUT_SEC, .tv_nsec > =3D 0}; > + int ret; > + > + MLX5_ASSERT(rte_eal_process_type() =3D=3D RTE_PROC_SECONDARY); > + mp_init_msg(mp_id, &mp_req, MLX5_MP_REQ_CREATE_MR); > + req->args.addr =3D addr; > + ret =3D rte_mp_request_sync(&mp_req, &mp_rep, &ts); > + if (ret) { > + DRV_LOG(ERR, "port %u request to primary process failed", > + mp_id->port_id); > + return -rte_errno; > + } > + MLX5_ASSERT(mp_rep.nb_received =3D=3D 1); > + mp_res =3D &mp_rep.msgs[0]; > + res =3D (struct mlx5_mp_param *)mp_res->param; > + ret =3D res->result; > + if (ret) > + rte_errno =3D -ret; > + free(mp_rep.msgs); > + return ret; > +} > + > +/** > + * Request Verbs queue state modification to the primary process. > + * > + * @param[in] mp_id > + * ID of the MP process. > + * @param sm > + * State modify parameters. > + * > + * @return > + * 0 on success, a negative errno value otherwise and rte_errno is set= . > + */ > +int > +mlx5_mp_req_queue_state_modify(struct mlx5_mp_id *mp_id, > + struct mlx5_mp_arg_queue_state_modify *sm) { > + struct rte_mp_msg mp_req; > + struct rte_mp_msg *mp_res; > + struct rte_mp_reply mp_rep; > + struct mlx5_mp_param *req =3D (struct mlx5_mp_param > *)mp_req.param; > + struct mlx5_mp_param *res; > + struct timespec ts =3D {.tv_sec =3D MLX5_MP_REQ_TIMEOUT_SEC, .tv_nsec > =3D 0}; > + int ret; > + > + MLX5_ASSERT(rte_eal_process_type() =3D=3D RTE_PROC_SECONDARY); > + mp_init_msg(mp_id, &mp_req, > MLX5_MP_REQ_QUEUE_STATE_MODIFY); > + req->args.state_modify =3D *sm; > + ret =3D rte_mp_request_sync(&mp_req, &mp_rep, &ts); > + if (ret) { > + DRV_LOG(ERR, "port %u request to primary process failed", > + mp_id->port_id); > + return -rte_errno; > + } > + MLX5_ASSERT(mp_rep.nb_received =3D=3D 1); > + mp_res =3D &mp_rep.msgs[0]; > + res =3D (struct mlx5_mp_param *)mp_res->param; > + ret =3D res->result; > + free(mp_rep.msgs); > + return ret; > +} > + > +/** > + * Request Verbs command file descriptor for mmap to the primary process= . > + * > + * @param[in] mp_id > + * ID of the MP process. > + * > + * @return > + * fd on success, a negative errno value otherwise and rte_errno is se= t. > + */ > +int > +mlx5_mp_req_verbs_cmd_fd(struct mlx5_mp_id *mp_id) { > + struct rte_mp_msg mp_req; > + struct rte_mp_msg *mp_res; > + struct rte_mp_reply mp_rep; > + struct mlx5_mp_param *res; > + struct timespec ts =3D {.tv_sec =3D MLX5_MP_REQ_TIMEOUT_SEC, .tv_nsec > =3D 0}; > + int ret; > + > + MLX5_ASSERT(rte_eal_process_type() =3D=3D RTE_PROC_SECONDARY); > + mp_init_msg(mp_id, &mp_req, MLX5_MP_REQ_VERBS_CMD_FD); > + ret =3D rte_mp_request_sync(&mp_req, &mp_rep, &ts); > + if (ret) { > + DRV_LOG(ERR, "port %u request to primary process failed", > + mp_id->port_id); > + return -rte_errno; > + } > + MLX5_ASSERT(mp_rep.nb_received =3D=3D 1); > + mp_res =3D &mp_rep.msgs[0]; > + res =3D (struct mlx5_mp_param *)mp_res->param; > + if (res->result) { > + rte_errno =3D -res->result; > + DRV_LOG(ERR, > + "port %u failed to get command FD from primary > process", > + mp_id->port_id); > + ret =3D -rte_errno; > + goto exit; > + } > + MLX5_ASSERT(mp_res->num_fds =3D=3D 1); > + ret =3D mp_res->fds[0]; > + DRV_LOG(DEBUG, "port %u command FD from primary is %d", > + mp_id->port_id, ret); > +exit: > + free(mp_rep.msgs); > + return ret; > +} > + > +/** > + * Initialize by primary process. > + */ > +int > +mlx5_mp_init_primary(const char *name, const rte_mp_t primary_action) { > + int ret; > + > + MLX5_ASSERT(rte_eal_process_type() =3D=3D RTE_PROC_PRIMARY); > + > + /* primary is allowed to not support IPC */ > + ret =3D rte_mp_action_register(name, primary_action); > + if (ret && rte_errno !=3D ENOTSUP) > + return -1; > + return 0; > +} > + > +/** > + * Un-initialize by primary process. > + */ > +void > +mlx5_mp_uninit_primary(const char *name) { > + MLX5_ASSERT(rte_eal_process_type() =3D=3D RTE_PROC_PRIMARY); > + rte_mp_action_unregister(name); > +} > + > +/** > + * Initialize by secondary process. > + */ > +int > +mlx5_mp_init_secondary(const char *name, const rte_mp_t > +secondary_action) { > + MLX5_ASSERT(rte_eal_process_type() =3D=3D RTE_PROC_SECONDARY); > + return rte_mp_action_register(name, secondary_action); } > + > +/** > + * Un-initialize by secondary process. > + */ > +void > +mlx5_mp_uninit_secondary(const char *name) { > + MLX5_ASSERT(rte_eal_process_type() =3D=3D RTE_PROC_SECONDARY); > + rte_mp_action_unregister(name); > +} > diff --git a/drivers/common/mlx5/mlx5_common_mp.h > b/drivers/common/mlx5/mlx5_common_mp.h > new file mode 100644 > index 0000000000..7aab77acb2 > --- /dev/null > +++ b/drivers/common/mlx5/mlx5_common_mp.h > @@ -0,0 +1,98 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright 2018 6WIND S.A. > + * Copyright 2018 Mellanox Technologies, Ltd */ > + > +#ifndef RTE_PMD_MLX5_COMMON_MP_H_ > +#define RTE_PMD_MLX5_COMMON_MP_H_ > + > +/* Verbs header. */ > +/* ISO C doesn't support unnamed structs/unions, disabling -pedantic. > +*/ #ifdef PEDANTIC #pragma GCC diagnostic ignored "-Wpedantic" > +#endif > +#include > +#ifdef PEDANTIC > +#pragma GCC diagnostic error "-Wpedantic" > +#endif > + > +#include > +#include > + > +/* Request types for IPC. */ > +enum mlx5_mp_req_type { > + MLX5_MP_REQ_VERBS_CMD_FD =3D 1, > + MLX5_MP_REQ_CREATE_MR, > + MLX5_MP_REQ_START_RXTX, > + MLX5_MP_REQ_STOP_RXTX, > + MLX5_MP_REQ_QUEUE_STATE_MODIFY, > +}; > + > +struct mlx5_mp_arg_queue_state_modify { > + uint8_t is_wq; /* Set if WQ. */ > + uint16_t queue_id; /* DPDK queue ID. */ > + enum ibv_wq_state state; /* WQ requested state. */ }; > + > +/* Pameters for IPC. */ > +struct mlx5_mp_param { > + enum mlx5_mp_req_type type; > + int port_id; > + int result; > + RTE_STD_C11 > + union { > + uintptr_t addr; /* MLX5_MP_REQ_CREATE_MR */ > + struct mlx5_mp_arg_queue_state_modify state_modify; > + /* MLX5_MP_REQ_QUEUE_STATE_MODIFY */ > + } args; > +}; > + > +/* Identifier of a MP process */ > +struct mlx5_mp_id { > + char name[RTE_MP_MAX_NAME_LEN]; > + uint16_t port_id; > +}; > + > +/** Request timeout for IPC. */ > +#define MLX5_MP_REQ_TIMEOUT_SEC 5 > + > +/** > + * Initialize IPC message. > + * > + * @param[in] port_id > + * Port ID of the device. > + * @param[out] msg > + * Pointer to message to fill in. > + * @param[in] type > + * Message type. > + */ > +static inline void > +mp_init_msg(struct mlx5_mp_id *mp_id, struct rte_mp_msg *msg, > + enum mlx5_mp_req_type type) > +{ > + struct mlx5_mp_param *param =3D (struct mlx5_mp_param *)msg- > >param; > + > + memset(msg, 0, sizeof(*msg)); > + strlcpy(msg->name, mp_id->name, sizeof(msg->name)); > + msg->len_param =3D sizeof(*param); > + param->type =3D type; > + param->port_id =3D mp_id->port_id; > +} > + > +__rte_experimental > +int mlx5_mp_init_primary(const char *name, const rte_mp_t > +primary_action); __rte_experimental void mlx5_mp_uninit_primary(const > +char *name); __rte_experimental int mlx5_mp_init_secondary(const char > +*name, const rte_mp_t secondary_action); __rte_experimental void > +mlx5_mp_uninit_secondary(const char *name); __rte_experimental int > +mlx5_mp_req_mr_create(struct mlx5_mp_id *mp_id, uintptr_t addr); > +__rte_experimental int mlx5_mp_req_queue_state_modify(struct > mlx5_mp_id > +*mp_id, > + struct mlx5_mp_arg_queue_state_modify > *sm); __rte_experimental > +int mlx5_mp_req_verbs_cmd_fd(struct mlx5_mp_id *mp_id); > + > +#endif /* RTE_PMD_MLX5_COMMON_MP_H_ */ > diff --git a/drivers/common/mlx5/rte_common_mlx5_version.map > b/drivers/common/mlx5/rte_common_mlx5_version.map > index aede2a0a51..265703d1c9 100644 > --- a/drivers/common/mlx5/rte_common_mlx5_version.map > +++ b/drivers/common/mlx5/rte_common_mlx5_version.map > @@ -48,4 +48,17 @@ DPDK_20.0.1 { > mlx5_nl_vlan_vmwa_delete; >=20 > mlx5_translate_port_name; > + > +}; > + > +EXPERIMENTAL { > + global: > + > + mlx5_mp_init_primary; > + mlx5_mp_uninit_primary; > + mlx5_mp_init_secondary; > + mlx5_mp_uninit_secondary; > + mlx5_mp_req_mr_create; > + mlx5_mp_req_queue_state_modify; > + mlx5_mp_req_verbs_cmd_fd; > }; > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index > 293d316413..d87c384422 100644 > --- a/drivers/net/mlx5/mlx5.c > +++ b/drivers/net/mlx5/mlx5.c > @@ -38,6 +38,7 @@ > #include > #include > #include > +#include >=20 > #include "mlx5_defs.h" > #include "mlx5.h" > @@ -1722,7 +1723,8 @@ mlx5_init_once(void) > rte_rwlock_init(&sd->mem_event_rwlock); > rte_mem_event_callback_register("MLX5_MEM_EVENT_CB", > mlx5_mr_mem_event_cb, > NULL); > - ret =3D mlx5_mp_init_primary(); > + ret =3D mlx5_mp_init_primary(MLX5_MP_NAME, > + mlx5_mp_primary_handle); > if (ret) > goto out; > sd->init_done =3D true; > @@ -1730,7 +1732,8 @@ mlx5_init_once(void) > case RTE_PROC_SECONDARY: > if (ld->init_done) > break; > - ret =3D mlx5_mp_init_secondary(); > + ret =3D mlx5_mp_init_secondary(MLX5_MP_NAME, > + mlx5_mp_secondary_handle); > if (ret) > goto out; > ++sd->secondary_cnt; > @@ -2205,6 +2208,8 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, > } > DRV_LOG(DEBUG, "naming Ethernet device \"%s\"", name); > if (rte_eal_process_type() =3D=3D RTE_PROC_SECONDARY) { > + struct mlx5_mp_id mp_id; > + > eth_dev =3D rte_eth_dev_attach_secondary(name); > if (eth_dev =3D=3D NULL) { > DRV_LOG(ERR, "can not attach rte ethdev"); @@ - > 2216,8 +2221,10 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, > err =3D mlx5_proc_priv_init(eth_dev); > if (err) > return NULL; > + mp_id.port_id =3D eth_dev->data->port_id; > + strlcpy(mp_id.name, MLX5_MP_NAME, > RTE_MP_MAX_NAME_LEN); > /* Receive command fd from primary process */ > - err =3D mlx5_mp_req_verbs_cmd_fd(eth_dev); > + err =3D mlx5_mp_req_verbs_cmd_fd(&mp_id); > if (err < 0) > return NULL; > /* Remap UAR for Tx queues. */ > @@ -2379,6 +2386,8 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, > priv->ibv_port =3D spawn->ibv_port; > priv->pci_dev =3D spawn->pci_dev; > priv->mtu =3D RTE_ETHER_MTU; > + priv->mp_id.port_id =3D port_id; > + strlcpy(priv->mp_id.name, MLX5_MP_NAME, > RTE_MP_MAX_NAME_LEN); > #ifndef RTE_ARCH_64 > /* Initialize UAR access locks for 32bit implementations. */ > rte_spinlock_init(&priv->uar_lock_cq); > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index > fccfe47341..e9d5868883 100644 > --- a/drivers/net/mlx5/mlx5.h > +++ b/drivers/net/mlx5/mlx5.h > @@ -36,43 +36,13 @@ > #include > #include > #include > +#include >=20 > #include "mlx5_defs.h" > #include "mlx5_utils.h" > #include "mlx5_mr.h" > #include "mlx5_autoconf.h" >=20 > -/* Request types for IPC. */ > -enum mlx5_mp_req_type { > - MLX5_MP_REQ_VERBS_CMD_FD =3D 1, > - MLX5_MP_REQ_CREATE_MR, > - MLX5_MP_REQ_START_RXTX, > - MLX5_MP_REQ_STOP_RXTX, > - MLX5_MP_REQ_QUEUE_STATE_MODIFY, > -}; > - > -struct mlx5_mp_arg_queue_state_modify { > - uint8_t is_wq; /* Set if WQ. */ > - uint16_t queue_id; /* DPDK queue ID. */ > - enum ibv_wq_state state; /* WQ requested state. */ > -}; > - > -/* Pameters for IPC. */ > -struct mlx5_mp_param { > - enum mlx5_mp_req_type type; > - int port_id; > - int result; > - RTE_STD_C11 > - union { > - uintptr_t addr; /* MLX5_MP_REQ_CREATE_MR */ > - struct mlx5_mp_arg_queue_state_modify state_modify; > - /* MLX5_MP_REQ_QUEUE_STATE_MODIFY */ > - } args; > -}; > - > -/** Request timeout for IPC. */ > -#define MLX5_MP_REQ_TIMEOUT_SEC 5 > - > /** Key string for IPC. */ > #define MLX5_MP_NAME "net_mlx5_mp" >=20 > @@ -583,6 +553,7 @@ struct mlx5_priv { > #endif > uint8_t skip_default_rss_reta; /* Skip configuration of default reta. *= / > uint8_t fdb_def_rule; /* Whether fdb jump to table 1 is configured. */ > + struct mlx5_mp_id mp_id; /* ID of a multi-process process */ > }; >=20 > #define PORT_ID(priv) ((priv)->dev_data->port_id) @@ -783,16 +754,10 @@ > int mlx5_flow_dev_dump(struct rte_eth_dev *dev, FILE *file, > struct rte_flow_error *error); >=20 > /* mlx5_mp.c */ > +int mlx5_mp_primary_handle(const struct rte_mp_msg *mp_msg, const void > +*peer); int mlx5_mp_secondary_handle(const struct rte_mp_msg *mp_msg, > +const void *peer); > void mlx5_mp_req_start_rxtx(struct rte_eth_dev *dev); void > mlx5_mp_req_stop_rxtx(struct rte_eth_dev *dev); -int > mlx5_mp_req_mr_create(struct rte_eth_dev *dev, uintptr_t addr); -int > mlx5_mp_req_verbs_cmd_fd(struct rte_eth_dev *dev); -int > mlx5_mp_req_queue_state_modify(struct rte_eth_dev *dev, > - struct mlx5_mp_arg_queue_state_modify > *sm); > -int mlx5_mp_init_primary(void); > -void mlx5_mp_uninit_primary(void); > -int mlx5_mp_init_secondary(void); > -void mlx5_mp_uninit_secondary(void); >=20 > /* mlx5_socket.c */ >=20 > diff --git a/drivers/net/mlx5/mlx5_mp.c b/drivers/net/mlx5/mlx5_mp.c inde= x > 55d408fe95..43684dbc3a 100644 > --- a/drivers/net/mlx5/mlx5_mp.c > +++ b/drivers/net/mlx5/mlx5_mp.c > @@ -10,46 +10,14 @@ > #include > #include >=20 > +#include > + > #include "mlx5.h" > #include "mlx5_rxtx.h" > #include "mlx5_utils.h" >=20 > -/** > - * Initialize IPC message. > - * > - * @param[in] dev > - * Pointer to Ethernet structure. > - * @param[out] msg > - * Pointer to message to fill in. > - * @param[in] type > - * Message type. > - */ > -static inline void > -mp_init_msg(struct rte_eth_dev *dev, struct rte_mp_msg *msg, > - enum mlx5_mp_req_type type) > -{ > - struct mlx5_mp_param *param =3D (struct mlx5_mp_param *)msg- > >param; > - > - memset(msg, 0, sizeof(*msg)); > - strlcpy(msg->name, MLX5_MP_NAME, sizeof(msg->name)); > - msg->len_param =3D sizeof(*param); > - param->type =3D type; > - param->port_id =3D dev->data->port_id; > -} > - > -/** > - * IPC message handler of primary process. > - * > - * @param[in] dev > - * Pointer to Ethernet structure. > - * @param[in] peer > - * Pointer to the peer socket path. > - * > - * @return > - * 0 on success, a negative errno value otherwise and rte_errno is set= . > - */ > -static int > -mp_primary_handle(const struct rte_mp_msg *mp_msg, const void *peer) > +int > +mlx5_mp_primary_handle(const struct rte_mp_msg *mp_msg, const void > +*peer) > { > struct rte_mp_msg mp_res; > struct mlx5_mp_param *res =3D (struct mlx5_mp_param > *)mp_res.param; @@ -71,21 +39,21 @@ mp_primary_handle(const struct > rte_mp_msg *mp_msg, const void *peer) > priv =3D dev->data->dev_private; > switch (param->type) { > case MLX5_MP_REQ_CREATE_MR: > - mp_init_msg(dev, &mp_res, param->type); > + mp_init_msg(&priv->mp_id, &mp_res, param->type); > lkey =3D mlx5_mr_create_primary(dev, &entry, param- > >args.addr); > if (lkey =3D=3D UINT32_MAX) > res->result =3D -rte_errno; > ret =3D rte_mp_reply(&mp_res, peer); > break; > case MLX5_MP_REQ_VERBS_CMD_FD: > - mp_init_msg(dev, &mp_res, param->type); > + mp_init_msg(&priv->mp_id, &mp_res, param->type); > mp_res.num_fds =3D 1; > mp_res.fds[0] =3D priv->sh->ctx->cmd_fd; > res->result =3D 0; > ret =3D rte_mp_reply(&mp_res, peer); > break; > case MLX5_MP_REQ_QUEUE_STATE_MODIFY: > - mp_init_msg(dev, &mp_res, param->type); > + mp_init_msg(&priv->mp_id, &mp_res, param->type); > res->result =3D mlx5_queue_state_modify_primary > (dev, ¶m->args.state_modify); > ret =3D rte_mp_reply(&mp_res, peer); > @@ -110,14 +78,15 @@ mp_primary_handle(const struct rte_mp_msg > *mp_msg, const void *peer) > * @return > * 0 on success, a negative errno value otherwise and rte_errno is set= . > */ > -static int > -mp_secondary_handle(const struct rte_mp_msg *mp_msg, const void *peer) > +int > +mlx5_mp_secondary_handle(const struct rte_mp_msg *mp_msg, const void > +*peer) > { > struct rte_mp_msg mp_res; > struct mlx5_mp_param *res =3D (struct mlx5_mp_param > *)mp_res.param; > const struct mlx5_mp_param *param =3D > (const struct mlx5_mp_param *)mp_msg->param; > struct rte_eth_dev *dev; > + struct mlx5_priv *priv; > int ret; >=20 > MLX5_ASSERT(rte_eal_process_type() =3D=3D RTE_PROC_SECONDARY); > @@ -127,13 +96,14 @@ mp_secondary_handle(const struct rte_mp_msg > *mp_msg, const void *peer) > return -rte_errno; > } > dev =3D &rte_eth_devices[param->port_id]; > + priv =3D dev->data->dev_private; > switch (param->type) { > case MLX5_MP_REQ_START_RXTX: > DRV_LOG(INFO, "port %u starting datapath", dev->data- > >port_id); > rte_mb(); > dev->rx_pkt_burst =3D mlx5_select_rx_function(dev); > dev->tx_pkt_burst =3D mlx5_select_tx_function(dev); > - mp_init_msg(dev, &mp_res, param->type); > + mp_init_msg(&priv->mp_id, &mp_res, param->type); > res->result =3D 0; > ret =3D rte_mp_reply(&mp_res, peer); > break; > @@ -142,7 +112,7 @@ mp_secondary_handle(const struct rte_mp_msg > *mp_msg, const void *peer) > dev->rx_pkt_burst =3D removed_rx_burst; > dev->tx_pkt_burst =3D removed_tx_burst; > rte_mb(); > - mp_init_msg(dev, &mp_res, param->type); > + mp_init_msg(&priv->mp_id, &mp_res, param->type); > res->result =3D 0; > ret =3D rte_mp_reply(&mp_res, peer); > break; > @@ -171,6 +141,7 @@ mp_req_on_rxtx(struct rte_eth_dev *dev, enum > mlx5_mp_req_type type) > struct rte_mp_reply mp_rep; > struct mlx5_mp_param *res; > struct timespec ts =3D {.tv_sec =3D MLX5_MP_REQ_TIMEOUT_SEC, .tv_nsec > =3D 0}; > + struct mlx5_priv *priv =3D dev->data->dev_private; > int ret; > int i; >=20 > @@ -182,7 +153,7 @@ mp_req_on_rxtx(struct rte_eth_dev *dev, enum > mlx5_mp_req_type type) > dev->data->port_id, type); > return; > } > - mp_init_msg(dev, &mp_req, type); > + mp_init_msg(&priv->mp_id, &mp_req, type); > ret =3D rte_mp_request_sync(&mp_req, &mp_rep, &ts); > if (ret) { > if (rte_errno !=3D ENOTSUP) > @@ -234,178 +205,3 @@ mlx5_mp_req_stop_rxtx(struct rte_eth_dev *dev) { > mp_req_on_rxtx(dev, MLX5_MP_REQ_STOP_RXTX); } > - > -/** > - * Request Memory Region creation to the primary process. > - * > - * @param[in] dev > - * Pointer to Ethernet structure. > - * @param addr > - * Target virtual address to register. > - * > - * @return > - * 0 on success, a negative errno value otherwise and rte_errno is set= . > - */ > -int > -mlx5_mp_req_mr_create(struct rte_eth_dev *dev, uintptr_t addr) -{ > - struct rte_mp_msg mp_req; > - struct rte_mp_msg *mp_res; > - struct rte_mp_reply mp_rep; > - struct mlx5_mp_param *req =3D (struct mlx5_mp_param > *)mp_req.param; > - struct mlx5_mp_param *res; > - struct timespec ts =3D {.tv_sec =3D MLX5_MP_REQ_TIMEOUT_SEC, .tv_nsec > =3D 0}; > - int ret; > - > - MLX5_ASSERT(rte_eal_process_type() =3D=3D RTE_PROC_SECONDARY); > - mp_init_msg(dev, &mp_req, MLX5_MP_REQ_CREATE_MR); > - req->args.addr =3D addr; > - ret =3D rte_mp_request_sync(&mp_req, &mp_rep, &ts); > - if (ret) { > - DRV_LOG(ERR, "port %u request to primary process failed", > - dev->data->port_id); > - return -rte_errno; > - } > - MLX5_ASSERT(mp_rep.nb_received =3D=3D 1); > - mp_res =3D &mp_rep.msgs[0]; > - res =3D (struct mlx5_mp_param *)mp_res->param; > - ret =3D res->result; > - if (ret) > - rte_errno =3D -ret; > - free(mp_rep.msgs); > - return ret; > -} > - > -/** > - * Request Verbs queue state modification to the primary process. > - * > - * @param[in] dev > - * Pointer to Ethernet structure. > - * @param sm > - * State modify parameters. > - * > - * @return > - * 0 on success, a negative errno value otherwise and rte_errno is set= . > - */ > -int > -mlx5_mp_req_queue_state_modify(struct rte_eth_dev *dev, > - struct mlx5_mp_arg_queue_state_modify *sm) > -{ > - struct rte_mp_msg mp_req; > - struct rte_mp_msg *mp_res; > - struct rte_mp_reply mp_rep; > - struct mlx5_mp_param *req =3D (struct mlx5_mp_param > *)mp_req.param; > - struct mlx5_mp_param *res; > - struct timespec ts =3D {.tv_sec =3D MLX5_MP_REQ_TIMEOUT_SEC, .tv_nsec > =3D 0}; > - int ret; > - > - MLX5_ASSERT(rte_eal_process_type() =3D=3D RTE_PROC_SECONDARY); > - mp_init_msg(dev, &mp_req, MLX5_MP_REQ_QUEUE_STATE_MODIFY); > - req->args.state_modify =3D *sm; > - ret =3D rte_mp_request_sync(&mp_req, &mp_rep, &ts); > - if (ret) { > - DRV_LOG(ERR, "port %u request to primary process failed", > - dev->data->port_id); > - return -rte_errno; > - } > - MLX5_ASSERT(mp_rep.nb_received =3D=3D 1); > - mp_res =3D &mp_rep.msgs[0]; > - res =3D (struct mlx5_mp_param *)mp_res->param; > - ret =3D res->result; > - free(mp_rep.msgs); > - return ret; > -} > - > -/** > - * Request Verbs command file descriptor for mmap to the primary process= . > - * > - * @param[in] dev > - * Pointer to Ethernet structure. > - * > - * @return > - * fd on success, a negative errno value otherwise and rte_errno is se= t. > - */ > -int > -mlx5_mp_req_verbs_cmd_fd(struct rte_eth_dev *dev) -{ > - struct rte_mp_msg mp_req; > - struct rte_mp_msg *mp_res; > - struct rte_mp_reply mp_rep; > - struct mlx5_mp_param *res; > - struct timespec ts =3D {.tv_sec =3D MLX5_MP_REQ_TIMEOUT_SEC, .tv_nsec > =3D 0}; > - int ret; > - > - MLX5_ASSERT(rte_eal_process_type() =3D=3D RTE_PROC_SECONDARY); > - mp_init_msg(dev, &mp_req, MLX5_MP_REQ_VERBS_CMD_FD); > - ret =3D rte_mp_request_sync(&mp_req, &mp_rep, &ts); > - if (ret) { > - DRV_LOG(ERR, "port %u request to primary process failed", > - dev->data->port_id); > - return -rte_errno; > - } > - MLX5_ASSERT(mp_rep.nb_received =3D=3D 1); > - mp_res =3D &mp_rep.msgs[0]; > - res =3D (struct mlx5_mp_param *)mp_res->param; > - if (res->result) { > - rte_errno =3D -res->result; > - DRV_LOG(ERR, > - "port %u failed to get command FD from primary > process", > - dev->data->port_id); > - ret =3D -rte_errno; > - goto exit; > - } > - MLX5_ASSERT(mp_res->num_fds =3D=3D 1); > - ret =3D mp_res->fds[0]; > - DRV_LOG(DEBUG, "port %u command FD from primary is %d", > - dev->data->port_id, ret); > -exit: > - free(mp_rep.msgs); > - return ret; > -} > - > -/** > - * Initialize by primary process. > - */ > -int > -mlx5_mp_init_primary(void) > -{ > - int ret; > - > - MLX5_ASSERT(rte_eal_process_type() =3D=3D RTE_PROC_PRIMARY); > - > - /* primary is allowed to not support IPC */ > - ret =3D rte_mp_action_register(MLX5_MP_NAME, mp_primary_handle); > - if (ret && rte_errno !=3D ENOTSUP) > - return -1; > - return 0; > -} > - > -/** > - * Un-initialize by primary process. > - */ > -void > -mlx5_mp_uninit_primary(void) > -{ > - MLX5_ASSERT(rte_eal_process_type() =3D=3D RTE_PROC_PRIMARY); > - rte_mp_action_unregister(MLX5_MP_NAME); > -} > - > -/** > - * Initialize by secondary process. > - */ > -int > -mlx5_mp_init_secondary(void) > -{ > - MLX5_ASSERT(rte_eal_process_type() =3D=3D RTE_PROC_SECONDARY); > - return rte_mp_action_register(MLX5_MP_NAME, > mp_secondary_handle); > -} > - > -/** > - * Un-initialize by secondary process. > - */ > -void > -mlx5_mp_uninit_secondary(void) > -{ > - MLX5_ASSERT(rte_eal_process_type() =3D=3D RTE_PROC_SECONDARY); > - rte_mp_action_unregister(MLX5_MP_NAME); > -} > diff --git a/drivers/net/mlx5/mlx5_mr.c b/drivers/net/mlx5/mlx5_mr.c inde= x > a8f185a208..9151992a72 100644 > --- a/drivers/net/mlx5/mlx5_mr.c > +++ b/drivers/net/mlx5/mlx5_mr.c > @@ -540,7 +540,7 @@ mlx5_mr_create_secondary(struct rte_eth_dev *dev, > struct mlx5_mr_cache *entry, >=20 > DEBUG("port %u requesting MR creation for address (%p)", > dev->data->port_id, (void *)addr); > - ret =3D mlx5_mp_req_mr_create(dev, addr); > + ret =3D mlx5_mp_req_mr_create(&priv->mp_id, addr); > if (ret) { > DEBUG("port %u fail to request MR creation for address > (%p)", > dev->data->port_id, (void *)addr); diff --git > a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index > 7ce3732fd3..42d7da8a4b 100644 > --- a/drivers/net/mlx5/mlx5_rxtx.c > +++ b/drivers/net/mlx5/mlx5_rxtx.c > @@ -1000,6 +1000,7 @@ static int > mlx5_queue_state_modify(struct rte_eth_dev *dev, > struct mlx5_mp_arg_queue_state_modify *sm) { > + struct mlx5_priv *priv =3D dev->data->dev_private; > int ret =3D 0; >=20 > switch (rte_eal_process_type()) { > @@ -1007,7 +1008,7 @@ mlx5_queue_state_modify(struct rte_eth_dev *dev, > ret =3D mlx5_queue_state_modify_primary(dev, sm); > break; > case RTE_PROC_SECONDARY: > - ret =3D mlx5_mp_req_queue_state_modify(dev, sm); > + ret =3D mlx5_mp_req_queue_state_modify(&priv->mp_id, sm); > break; > default: > break; > -- > 2.16.6