From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140051.outbound.protection.outlook.com [40.107.14.51]) by dpdk.org (Postfix) with ESMTP id 305341B486 for ; Thu, 21 Mar 2019 13:13:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NKNnPpw2Gt5nxIzNwvBtHn85yriK11d0/jS+H1+wxfQ=; b=c2dMlZqfPCz9aqh4d5b+IQOgMeEF6ZGQNnZeXC8uJwA90+j45Fmh/JS6vZFgwNXPvmZyO/FzhLLCUOg6DmyumkJ00pzYEnYtNWhJf1YAD8/qvNyXwWavCy6D/oSdZWvykMYcbfcQila6OI/3sc4ZO3xlkRQTC7/fX2nxzhk64f4= Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com (52.133.45.150) by AM0PR0502MB3827.eurprd05.prod.outlook.com (52.133.47.141) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1730.15; Thu, 21 Mar 2019 12:13:38 +0000 Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003]) by AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003%2]) with mapi id 15.20.1730.013; Thu, 21 Mar 2019 12:13:38 +0000 From: Shahaf Shuler To: Slava Ovsiienko , "dev@dpdk.org" Thread-Topic: [PATCH 00/14] net/mlx5: add support for multiport IB devices Thread-Index: AQHU373Fos9WaxsH5ky513ZDp4tBuqYV/1pQ Date: Thu, 21 Mar 2019 12:13:38 +0000 Message-ID: References: <1551376985-11096-1-git-send-email-viacheslavo@mellanox.com> <1553155888-27498-1-git-send-email-viacheslavo@mellanox.com> In-Reply-To: <1553155888-27498-1-git-send-email-viacheslavo@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=shahafs@mellanox.com; x-originating-ip: [31.154.10.105] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2467d1e3-881c-4766-f762-08d6adf6a692 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM0PR0502MB3827; x-ms-traffictypediagnostic: AM0PR0502MB3827: x-ms-exchange-purlcount: 1 x-microsoft-antispam-prvs: x-forefront-prvs: 0983EAD6B2 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(396003)(346002)(39860400002)(136003)(376002)(189003)(199004)(81166006)(97736004)(33656002)(68736007)(74316002)(25786009)(7736002)(305945005)(110136005)(316002)(14454004)(2906002)(446003)(2501003)(14444005)(52536014)(11346002)(476003)(71190400001)(256004)(5024004)(86362001)(486006)(99286004)(186003)(5660300002)(66066001)(26005)(7696005)(53936002)(76176011)(102836004)(55016002)(9686003)(6436002)(71200400001)(229853002)(6306002)(3846002)(966005)(8676002)(8936002)(81156014)(478600001)(6246003)(106356001)(6506007)(6116002)(105586002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR0502MB3827; H:AM0PR0502MB3795.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:3; A:3; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: Fg6GlElPHpXZF09wthEMBRsus94UtjVUfMmUC6x1pQYeNSymi6ZNqDG8vmvQeI5uJCm9yLHluk6q4I+2HJ2bgLa844U/5bHDYK1jd25YKHPEzIth1a22HOa3Q50pUbeLeJ7VyHVwl1p2H9Js7dui94lGiqvArTMHQwNI4vyso3kG3y9AeZvGgj3q7kfo2Z6jOmLKsIKhak7PKjn3xU8EVBX8vWJhLjNggIl/JpIHpOVAmTFqtDkF3QhpFS36yePGNuoCpSKNp0GzYyZdwzWtkbdyBDChFLAz5VFqFUO5MRYmCq8d6UD9CLxSh+zTXB0W+qtrpfCpYanqcl8ZHaHmbOsOkqDSLl/sBjr0iH+/yrtaXCQPgBbm0jfQD4oeNsBgxvmDCLHUJkhPx49Mk/ogXwFJ9uxoiCS1jo8CCQp2Z9M= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2467d1e3-881c-4766-f762-08d6adf6a692 X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Mar 2019 12:13:38.6592 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB3827 Subject: Re: [dpdk-dev] [PATCH 00/14] net/mlx5: add support for multiport IB devices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Mar 2019 12:13:42 -0000 Thursday, March 21, 2019 10:11 AM, Viacheslav Ovsiienko: > Subject: [PATCH 00/14] net/mlx5: add support for multiport IB devices >=20 > The Mellanox NICs support SR-IOV and have E-Switch feature. > When SR-IOV is set up in switchdev mode and E-Switch is enabled we have > so called VF representors in the system. All representors belonging to th= e > same E-Switch are created on the basis of the single PCI function and wit= h > current implementation each representor has its own dedicated Infiniband > device and operates within its own Infiniband context. It is proposed to > provide representors as ports of the single Infiniband device and operate= on > the shared Infiniband context. >=20 > mlx5_pci_probe() routine is planned to be extended to probe the ports of > found Infiniband devices. All active ports, belonging to the same Infinib= and > device will use the signle shared Infiniband context of that device: > - QP are created within shared context, with assigning to > specified port > - Verb flows are also created with specifying port index > - Protection Domain is shared by all representors > - Global Memory Regions Cache is shared by all representors, > local ones are not (not in this patchset) > - Shared Asynchronous Event Handler distributes events > among the device ports (not in this patchset) > - UAR mapping to user space is also shared (not in patchset) >=20 > Each representor has its own attach network interface device, interface > name and index can be retrieved via IB device and port index, so the link > control is performed via Netlink in the same way as in existing > implementation. Nice Work Slava, Please see my comments.=20 Also - where is the support for sharing of the memory regions?=20 >=20 > This patchset also superseedes http://patches.dpdk.org/patch/50486/ >=20 > Signed-off-by: Viacheslav Ovsiienko >=20 > Viacheslav Ovsiienko (14): > net/mlx5: add representor recognition on kernels 5.x > net/mlx5: introduce multiport IB device shared structure > net/mlx5: modify get ifindex routine for multiport IB > net/mlx5: add getting IB ports number for multiport IB > net/mlx5: add multiport IB device support to probing > net/mlx5: add IB shared context alloc/free functions > net/mlx5: switch to the names in the shared IB context > net/mlx5: switch to the shared Protection Domain > net/mlx5: switch to the shared context IB attributes > net/mlx5: switch to the shared IB device context > net/mlx5: provide IB port for the object being created > net/mlx5: update install/uninstall int handler routines > net/mlx5: update event handler for multiport IB devices > net/mlx5: add source vport match to the ingress rules >=20 > drivers/net/mlx5/Makefile | 10 + > drivers/net/mlx5/meson.build | 4 + > drivers/net/mlx5/mlx5.c | 549 +++++++++++++++++++++++++++----= -- > ---- > drivers/net/mlx5/mlx5.h | 38 ++- > drivers/net/mlx5/mlx5_ethdev.c | 241 ++++++++++------ > drivers/net/mlx5/mlx5_flow.c | 2 + > drivers/net/mlx5/mlx5_flow_dv.c | 47 +++- > drivers/net/mlx5/mlx5_flow_verbs.c | 7 +- > drivers/net/mlx5/mlx5_mr.c | 4 +- > drivers/net/mlx5/mlx5_nl.c | 101 ++++++- > drivers/net/mlx5/mlx5_rxq.c | 41 +-- > drivers/net/mlx5/mlx5_socket.c | 4 +- > drivers/net/mlx5/mlx5_stats.c | 22 +- > drivers/net/mlx5/mlx5_trigger.c | 2 +- > drivers/net/mlx5/mlx5_txq.c | 24 +- > 15 files changed, 795 insertions(+), 301 deletions(-) >=20 > -- > 1.8.3.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 43553A00E6 for ; Thu, 21 Mar 2019 13:13:45 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A48391B4AA; Thu, 21 Mar 2019 13:13:43 +0100 (CET) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140051.outbound.protection.outlook.com [40.107.14.51]) by dpdk.org (Postfix) with ESMTP id 305341B486 for ; Thu, 21 Mar 2019 13:13:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NKNnPpw2Gt5nxIzNwvBtHn85yriK11d0/jS+H1+wxfQ=; b=c2dMlZqfPCz9aqh4d5b+IQOgMeEF6ZGQNnZeXC8uJwA90+j45Fmh/JS6vZFgwNXPvmZyO/FzhLLCUOg6DmyumkJ00pzYEnYtNWhJf1YAD8/qvNyXwWavCy6D/oSdZWvykMYcbfcQila6OI/3sc4ZO3xlkRQTC7/fX2nxzhk64f4= Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com (52.133.45.150) by AM0PR0502MB3827.eurprd05.prod.outlook.com (52.133.47.141) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1730.15; Thu, 21 Mar 2019 12:13:38 +0000 Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003]) by AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003%2]) with mapi id 15.20.1730.013; Thu, 21 Mar 2019 12:13:38 +0000 From: Shahaf Shuler To: Slava Ovsiienko , "dev@dpdk.org" Thread-Topic: [PATCH 00/14] net/mlx5: add support for multiport IB devices Thread-Index: AQHU373Fos9WaxsH5ky513ZDp4tBuqYV/1pQ Date: Thu, 21 Mar 2019 12:13:38 +0000 Message-ID: References: <1551376985-11096-1-git-send-email-viacheslavo@mellanox.com> <1553155888-27498-1-git-send-email-viacheslavo@mellanox.com> In-Reply-To: <1553155888-27498-1-git-send-email-viacheslavo@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=shahafs@mellanox.com; x-originating-ip: [31.154.10.105] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2467d1e3-881c-4766-f762-08d6adf6a692 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM0PR0502MB3827; x-ms-traffictypediagnostic: AM0PR0502MB3827: x-ms-exchange-purlcount: 1 x-microsoft-antispam-prvs: x-forefront-prvs: 0983EAD6B2 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(396003)(346002)(39860400002)(136003)(376002)(189003)(199004)(81166006)(97736004)(33656002)(68736007)(74316002)(25786009)(7736002)(305945005)(110136005)(316002)(14454004)(2906002)(446003)(2501003)(14444005)(52536014)(11346002)(476003)(71190400001)(256004)(5024004)(86362001)(486006)(99286004)(186003)(5660300002)(66066001)(26005)(7696005)(53936002)(76176011)(102836004)(55016002)(9686003)(6436002)(71200400001)(229853002)(6306002)(3846002)(966005)(8676002)(8936002)(81156014)(478600001)(6246003)(106356001)(6506007)(6116002)(105586002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR0502MB3827; H:AM0PR0502MB3795.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:3; A:3; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: Fg6GlElPHpXZF09wthEMBRsus94UtjVUfMmUC6x1pQYeNSymi6ZNqDG8vmvQeI5uJCm9yLHluk6q4I+2HJ2bgLa844U/5bHDYK1jd25YKHPEzIth1a22HOa3Q50pUbeLeJ7VyHVwl1p2H9Js7dui94lGiqvArTMHQwNI4vyso3kG3y9AeZvGgj3q7kfo2Z6jOmLKsIKhak7PKjn3xU8EVBX8vWJhLjNggIl/JpIHpOVAmTFqtDkF3QhpFS36yePGNuoCpSKNp0GzYyZdwzWtkbdyBDChFLAz5VFqFUO5MRYmCq8d6UD9CLxSh+zTXB0W+qtrpfCpYanqcl8ZHaHmbOsOkqDSLl/sBjr0iH+/yrtaXCQPgBbm0jfQD4oeNsBgxvmDCLHUJkhPx49Mk/ogXwFJ9uxoiCS1jo8CCQp2Z9M= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2467d1e3-881c-4766-f762-08d6adf6a692 X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Mar 2019 12:13:38.6592 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB3827 Subject: Re: [dpdk-dev] [PATCH 00/14] net/mlx5: add support for multiport IB devices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190321121338.zSHcO8Dx4u11u2GO-l73_pBDlKJGtOpQxYwcX5REIVI@z> Thursday, March 21, 2019 10:11 AM, Viacheslav Ovsiienko: > Subject: [PATCH 00/14] net/mlx5: add support for multiport IB devices >=20 > The Mellanox NICs support SR-IOV and have E-Switch feature. > When SR-IOV is set up in switchdev mode and E-Switch is enabled we have > so called VF representors in the system. All representors belonging to th= e > same E-Switch are created on the basis of the single PCI function and wit= h > current implementation each representor has its own dedicated Infiniband > device and operates within its own Infiniband context. It is proposed to > provide representors as ports of the single Infiniband device and operate= on > the shared Infiniband context. >=20 > mlx5_pci_probe() routine is planned to be extended to probe the ports of > found Infiniband devices. All active ports, belonging to the same Infinib= and > device will use the signle shared Infiniband context of that device: > - QP are created within shared context, with assigning to > specified port > - Verb flows are also created with specifying port index > - Protection Domain is shared by all representors > - Global Memory Regions Cache is shared by all representors, > local ones are not (not in this patchset) > - Shared Asynchronous Event Handler distributes events > among the device ports (not in this patchset) > - UAR mapping to user space is also shared (not in patchset) >=20 > Each representor has its own attach network interface device, interface > name and index can be retrieved via IB device and port index, so the link > control is performed via Netlink in the same way as in existing > implementation. Nice Work Slava, Please see my comments.=20 Also - where is the support for sharing of the memory regions?=20 >=20 > This patchset also superseedes http://patches.dpdk.org/patch/50486/ >=20 > Signed-off-by: Viacheslav Ovsiienko >=20 > Viacheslav Ovsiienko (14): > net/mlx5: add representor recognition on kernels 5.x > net/mlx5: introduce multiport IB device shared structure > net/mlx5: modify get ifindex routine for multiport IB > net/mlx5: add getting IB ports number for multiport IB > net/mlx5: add multiport IB device support to probing > net/mlx5: add IB shared context alloc/free functions > net/mlx5: switch to the names in the shared IB context > net/mlx5: switch to the shared Protection Domain > net/mlx5: switch to the shared context IB attributes > net/mlx5: switch to the shared IB device context > net/mlx5: provide IB port for the object being created > net/mlx5: update install/uninstall int handler routines > net/mlx5: update event handler for multiport IB devices > net/mlx5: add source vport match to the ingress rules >=20 > drivers/net/mlx5/Makefile | 10 + > drivers/net/mlx5/meson.build | 4 + > drivers/net/mlx5/mlx5.c | 549 +++++++++++++++++++++++++++----= -- > ---- > drivers/net/mlx5/mlx5.h | 38 ++- > drivers/net/mlx5/mlx5_ethdev.c | 241 ++++++++++------ > drivers/net/mlx5/mlx5_flow.c | 2 + > drivers/net/mlx5/mlx5_flow_dv.c | 47 +++- > drivers/net/mlx5/mlx5_flow_verbs.c | 7 +- > drivers/net/mlx5/mlx5_mr.c | 4 +- > drivers/net/mlx5/mlx5_nl.c | 101 ++++++- > drivers/net/mlx5/mlx5_rxq.c | 41 +-- > drivers/net/mlx5/mlx5_socket.c | 4 +- > drivers/net/mlx5/mlx5_stats.c | 22 +- > drivers/net/mlx5/mlx5_trigger.c | 2 +- > drivers/net/mlx5/mlx5_txq.c | 24 +- > 15 files changed, 795 insertions(+), 301 deletions(-) >=20 > -- > 1.8.3.1