From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <shahafs@mellanox.com> Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50044.outbound.protection.outlook.com [40.107.5.44]) by dpdk.org (Postfix) with ESMTP id 72EE85689 for <dev@dpdk.org>; Wed, 27 Mar 2019 07:00:10 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SvrWBmhzkCeZpzMfN95U9AzNeXWs265UupkMVmE8uJs=; b=uASga0re8l8u5jVWYg3T+R3WRj+WV2NTBktXdSBFnNoHNnGfgqboLj4iLXLl1PRgD5YVEnOBjPFF/pyymjZVT9fAuc37ClhrnSdrIQgg33S0tbt+AepUtDzsQpFIfGamE2eRiuyg97HC0FyShQkCgFysR4rakEJSED+gKfGhTIk= Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com (52.133.45.150) by AM0PR0502MB4066.eurprd05.prod.outlook.com (52.133.41.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1730.16; Wed, 27 Mar 2019 06:00:06 +0000 Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003]) by AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003%2]) with mapi id 15.20.1750.014; Wed, 27 Mar 2019 06:00:06 +0000 From: Shahaf Shuler <shahafs@mellanox.com> To: Slava Ovsiienko <viacheslavo@mellanox.com>, "dev@dpdk.org" <dev@dpdk.org> Thread-Topic: [PATCH v3 00/14] net/mlx5: add support for multiport IB devices Thread-Index: AQHU4+muuPpyZQWfek6S5giQ62hsaqYe/GDQ Date: Wed, 27 Mar 2019 06:00:06 +0000 Message-ID: <AM0PR0502MB3795E581DDD624CCDB7FF08CC3580@AM0PR0502MB3795.eurprd05.prod.outlook.com> References: <1553533414-9911-1-git-send-email-viacheslavo@mellanox.com> <1553614522-12151-1-git-send-email-viacheslavo@mellanox.com> In-Reply-To: <1553614522-12151-1-git-send-email-viacheslavo@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=shahafs@mellanox.com; x-originating-ip: [31.154.10.105] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 441ca947-b639-41d7-57c0-08d6b2797642 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM0PR0502MB4066; x-ms-traffictypediagnostic: AM0PR0502MB4066: x-ms-exchange-purlcount: 4 x-microsoft-antispam-prvs: <AM0PR0502MB4066F00B96512AE3CBBCEBF4C3580@AM0PR0502MB4066.eurprd05.prod.outlook.com> x-forefront-prvs: 0989A7979C x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(136003)(39850400004)(366004)(376002)(346002)(189003)(199004)(6436002)(476003)(305945005)(2906002)(9686003)(11346002)(6306002)(478600001)(6116002)(7736002)(3846002)(966005)(110136005)(76176011)(446003)(186003)(74316002)(8676002)(55016002)(53936002)(66066001)(7696005)(71200400001)(68736007)(97736004)(8936002)(486006)(105586002)(2501003)(99286004)(6506007)(102836004)(5660300002)(14454004)(25786009)(71190400001)(316002)(256004)(81156014)(81166006)(229853002)(86362001)(6246003)(106356001)(52536014)(5024004)(26005)(14444005)(33656002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR0502MB4066; H:AM0PR0502MB3795.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: 0boXClManepPbjoJYQ5kpbmCkIvBASyDUEsTgOg/E0UpRLkS4ygzn1HSIrRb+JbxA3SUo2hzMSmf7I7fmRrRU0lrwtAN+/Vm7MJIGEuQOWPszKO/YuFVqWqe7lEfze/nAWKcqZOmRW7qvE07ueCdvCsU334P7VHd82RCIkmiuzqLlnBX4ZYch42foA/hmwakYSxD4xD8i6pJETcs9KcWEzK9dDziGOvq84nYHl57JW8EkzwhOrLRMHaSGc0RWah2JsWutVYNQpcJ8NHc4Z2GZ+49d7WRwhRV950+bcLQkepwz93qMznticTWZKKEla1ROIIVjR/8fb0KwOT5+f08vb3kP59yu4A3p1QPNp0HZC9sjW5jvFmWURYDwExt/cYn7itczaMscVbpf3Nrxp4YuIkXT2l6It0ZmWNsogk6hE0= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 441ca947-b639-41d7-57c0-08d6b2797642 X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Mar 2019 06:00:06.1908 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB4066 Subject: Re: [dpdk-dev] [PATCH v3 00/14] net/mlx5: add support for multiport IB devices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> X-List-Received-Date: Wed, 27 Mar 2019 06:00:10 -0000 Tuesday, March 26, 2019 5:35 PM, Viacheslav Ovsiienko: > Subject: [PATCH v3 00/14] net/mlx5: add support for multiport IB devices >=20 > The Mellanox NICs support SR-IOV and have E-Switch feature. > When SR-IOV is set up in switchdev mode and E-Switch is enabled we have > so called VF representors in the system. All representors belonging to th= e > same E-Switch are created on the basis of the single PCI function and wit= h > current implementation each representor has its own dedicated Infiniband > device and operates within its own Infiniband context. It is proposed to > provide representors as ports of the single Infiniband device and operate= on > the shared Infiniband context. >=20 > mlx5_pci_probe() routine is planned to be extended to probe the ports of > found Infiniband devices. All active ports, belonging to the same Infinib= and > device will use the signle shared Infiniband context of that device: > - QP are created within shared context, with assigning to > specified port > - Verb flows are also created with specifying port index > - Protection Domain is shared by all representors > - Global Memory Regions Cache is shared by all representors, > local ones are not (not in this patchset) > - Shared Asynchronous Event Handler distributes events > among the device ports (not in this patchset) > - UAR mapping to user space is also shared (not in patchset) >=20 > Each representor has its own attached network interface device, interface > name and index can be retrieved via IB device and port index, so the link > control is performed via Netlink in the same way as in existing > implementation. >=20 > This patchset also superseedes http://patches.dpdk.org/patch/50486/ >=20 > Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com> Slava, basically I am OK w/ the series, in order for me to apply: 1. need to rebase on top of latest master. There are some merge conflicts. 2. at least patch[1] seems to be taken from another branch (checkout the HA= VE_MLX5DV_DR flag). Please handle.=20 [1] https://patches.dpdk.org/patch/51739/ >=20 > --- > v3: > - vport_id field is added, contains associated VF vport > index on E-Switch. representor_id field is reverted > to keep the VF index > - some comment spellcheck > - some error logging is added for master/representor > wrong recognition > - shared IB context alloc/free routines mutex > protection added >=20 > v2: > http://patches.dpdk.org/patch/51642/ > - clarifying comments added > - secondary process does not probe IB device at all and > does not create shared context > - minor bugs (like ++ in assert) fixed > - representor_id contains corresponding VF vport index, > no corrections +1 is needed anymore. vport index is > calculated in the only place in code and if indexing > schema is changed it would be easy to modify >=20 > v1: > http://patches.dpdk.org/cover/51451/ >=20 > Viacheslav Ovsiienko (13): > net/mlx5: add representor recognition on kernels 5.x > net/mlx5: modify get ifindex routine for multiport IB > net/mlx5: add getting IB ports number for multiport IB > net/mlx5: add multiport IB device support to probing > net/mlx5: add IB shared context alloc/free functions > net/mlx5: switch to the names in the shared IB context > net/mlx5: switch to the shared Protection Domain > net/mlx5: switch to the shared context IB attributes > net/mlx5: switch to the shared IB device context > net/mlx5: provide IB port for the object being created > net/mlx5: update install/uninstall int handler routines > net/mlx5: update event handler for multiport IB devices > net/mlx5: add source vport match to the ingress rules >=20 > drivers/net/mlx5/Makefile | 10 + > drivers/net/mlx5/meson.build | 4 + > drivers/net/mlx5/mlx5.c | 651 ++++++++++++++++++++++++++-----= --- > --- > drivers/net/mlx5/mlx5.h | 45 ++- > drivers/net/mlx5/mlx5_ethdev.c | 248 +++++++++----- > drivers/net/mlx5/mlx5_flow.c | 2 + > drivers/net/mlx5/mlx5_flow_dv.c | 45 ++- > drivers/net/mlx5/mlx5_flow_verbs.c | 7 +- > drivers/net/mlx5/mlx5_mr.c | 4 +- > drivers/net/mlx5/mlx5_nl.c | 108 +++++- > drivers/net/mlx5/mlx5_rxq.c | 41 +-- > drivers/net/mlx5/mlx5_socket.c | 4 +- > drivers/net/mlx5/mlx5_stats.c | 22 +- > drivers/net/mlx5/mlx5_trigger.c | 2 +- > drivers/net/mlx5/mlx5_txq.c | 24 +- > 15 files changed, 879 insertions(+), 338 deletions(-) >=20 > -- > 1.8.3.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 28C80A05D3 for <public@inbox.dpdk.org>; Wed, 27 Mar 2019 07:00:13 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C8CFE569B; Wed, 27 Mar 2019 07:00:11 +0100 (CET) Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50044.outbound.protection.outlook.com [40.107.5.44]) by dpdk.org (Postfix) with ESMTP id 72EE85689 for <dev@dpdk.org>; Wed, 27 Mar 2019 07:00:10 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SvrWBmhzkCeZpzMfN95U9AzNeXWs265UupkMVmE8uJs=; b=uASga0re8l8u5jVWYg3T+R3WRj+WV2NTBktXdSBFnNoHNnGfgqboLj4iLXLl1PRgD5YVEnOBjPFF/pyymjZVT9fAuc37ClhrnSdrIQgg33S0tbt+AepUtDzsQpFIfGamE2eRiuyg97HC0FyShQkCgFysR4rakEJSED+gKfGhTIk= Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com (52.133.45.150) by AM0PR0502MB4066.eurprd05.prod.outlook.com (52.133.41.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1730.16; Wed, 27 Mar 2019 06:00:06 +0000 Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003]) by AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003%2]) with mapi id 15.20.1750.014; Wed, 27 Mar 2019 06:00:06 +0000 From: Shahaf Shuler <shahafs@mellanox.com> To: Slava Ovsiienko <viacheslavo@mellanox.com>, "dev@dpdk.org" <dev@dpdk.org> Thread-Topic: [PATCH v3 00/14] net/mlx5: add support for multiport IB devices Thread-Index: AQHU4+muuPpyZQWfek6S5giQ62hsaqYe/GDQ Date: Wed, 27 Mar 2019 06:00:06 +0000 Message-ID: <AM0PR0502MB3795E581DDD624CCDB7FF08CC3580@AM0PR0502MB3795.eurprd05.prod.outlook.com> References: <1553533414-9911-1-git-send-email-viacheslavo@mellanox.com> <1553614522-12151-1-git-send-email-viacheslavo@mellanox.com> In-Reply-To: <1553614522-12151-1-git-send-email-viacheslavo@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=shahafs@mellanox.com; x-originating-ip: [31.154.10.105] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 441ca947-b639-41d7-57c0-08d6b2797642 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM0PR0502MB4066; x-ms-traffictypediagnostic: AM0PR0502MB4066: x-ms-exchange-purlcount: 4 x-microsoft-antispam-prvs: <AM0PR0502MB4066F00B96512AE3CBBCEBF4C3580@AM0PR0502MB4066.eurprd05.prod.outlook.com> x-forefront-prvs: 0989A7979C x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(136003)(39850400004)(366004)(376002)(346002)(189003)(199004)(6436002)(476003)(305945005)(2906002)(9686003)(11346002)(6306002)(478600001)(6116002)(7736002)(3846002)(966005)(110136005)(76176011)(446003)(186003)(74316002)(8676002)(55016002)(53936002)(66066001)(7696005)(71200400001)(68736007)(97736004)(8936002)(486006)(105586002)(2501003)(99286004)(6506007)(102836004)(5660300002)(14454004)(25786009)(71190400001)(316002)(256004)(81156014)(81166006)(229853002)(86362001)(6246003)(106356001)(52536014)(5024004)(26005)(14444005)(33656002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR0502MB4066; H:AM0PR0502MB3795.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: 0boXClManepPbjoJYQ5kpbmCkIvBASyDUEsTgOg/E0UpRLkS4ygzn1HSIrRb+JbxA3SUo2hzMSmf7I7fmRrRU0lrwtAN+/Vm7MJIGEuQOWPszKO/YuFVqWqe7lEfze/nAWKcqZOmRW7qvE07ueCdvCsU334P7VHd82RCIkmiuzqLlnBX4ZYch42foA/hmwakYSxD4xD8i6pJETcs9KcWEzK9dDziGOvq84nYHl57JW8EkzwhOrLRMHaSGc0RWah2JsWutVYNQpcJ8NHc4Z2GZ+49d7WRwhRV950+bcLQkepwz93qMznticTWZKKEla1ROIIVjR/8fb0KwOT5+f08vb3kP59yu4A3p1QPNp0HZC9sjW5jvFmWURYDwExt/cYn7itczaMscVbpf3Nrxp4YuIkXT2l6It0ZmWNsogk6hE0= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 441ca947-b639-41d7-57c0-08d6b2797642 X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Mar 2019 06:00:06.1908 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB4066 Subject: Re: [dpdk-dev] [PATCH v3 00/14] net/mlx5: add support for multiport IB devices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> Message-ID: <20190327060006.Rw2YEkqX6M63rCLds2EtIGp0z4KXeNXnSaEJMeLXuVI@z> Tuesday, March 26, 2019 5:35 PM, Viacheslav Ovsiienko: > Subject: [PATCH v3 00/14] net/mlx5: add support for multiport IB devices >=20 > The Mellanox NICs support SR-IOV and have E-Switch feature. > When SR-IOV is set up in switchdev mode and E-Switch is enabled we have > so called VF representors in the system. All representors belonging to th= e > same E-Switch are created on the basis of the single PCI function and wit= h > current implementation each representor has its own dedicated Infiniband > device and operates within its own Infiniband context. It is proposed to > provide representors as ports of the single Infiniband device and operate= on > the shared Infiniband context. >=20 > mlx5_pci_probe() routine is planned to be extended to probe the ports of > found Infiniband devices. All active ports, belonging to the same Infinib= and > device will use the signle shared Infiniband context of that device: > - QP are created within shared context, with assigning to > specified port > - Verb flows are also created with specifying port index > - Protection Domain is shared by all representors > - Global Memory Regions Cache is shared by all representors, > local ones are not (not in this patchset) > - Shared Asynchronous Event Handler distributes events > among the device ports (not in this patchset) > - UAR mapping to user space is also shared (not in patchset) >=20 > Each representor has its own attached network interface device, interface > name and index can be retrieved via IB device and port index, so the link > control is performed via Netlink in the same way as in existing > implementation. >=20 > This patchset also superseedes http://patches.dpdk.org/patch/50486/ >=20 > Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com> Slava, basically I am OK w/ the series, in order for me to apply: 1. need to rebase on top of latest master. There are some merge conflicts. 2. at least patch[1] seems to be taken from another branch (checkout the HA= VE_MLX5DV_DR flag). Please handle.=20 [1] https://patches.dpdk.org/patch/51739/ >=20 > --- > v3: > - vport_id field is added, contains associated VF vport > index on E-Switch. representor_id field is reverted > to keep the VF index > - some comment spellcheck > - some error logging is added for master/representor > wrong recognition > - shared IB context alloc/free routines mutex > protection added >=20 > v2: > http://patches.dpdk.org/patch/51642/ > - clarifying comments added > - secondary process does not probe IB device at all and > does not create shared context > - minor bugs (like ++ in assert) fixed > - representor_id contains corresponding VF vport index, > no corrections +1 is needed anymore. vport index is > calculated in the only place in code and if indexing > schema is changed it would be easy to modify >=20 > v1: > http://patches.dpdk.org/cover/51451/ >=20 > Viacheslav Ovsiienko (13): > net/mlx5: add representor recognition on kernels 5.x > net/mlx5: modify get ifindex routine for multiport IB > net/mlx5: add getting IB ports number for multiport IB > net/mlx5: add multiport IB device support to probing > net/mlx5: add IB shared context alloc/free functions > net/mlx5: switch to the names in the shared IB context > net/mlx5: switch to the shared Protection Domain > net/mlx5: switch to the shared context IB attributes > net/mlx5: switch to the shared IB device context > net/mlx5: provide IB port for the object being created > net/mlx5: update install/uninstall int handler routines > net/mlx5: update event handler for multiport IB devices > net/mlx5: add source vport match to the ingress rules >=20 > drivers/net/mlx5/Makefile | 10 + > drivers/net/mlx5/meson.build | 4 + > drivers/net/mlx5/mlx5.c | 651 ++++++++++++++++++++++++++-----= --- > --- > drivers/net/mlx5/mlx5.h | 45 ++- > drivers/net/mlx5/mlx5_ethdev.c | 248 +++++++++----- > drivers/net/mlx5/mlx5_flow.c | 2 + > drivers/net/mlx5/mlx5_flow_dv.c | 45 ++- > drivers/net/mlx5/mlx5_flow_verbs.c | 7 +- > drivers/net/mlx5/mlx5_mr.c | 4 +- > drivers/net/mlx5/mlx5_nl.c | 108 +++++- > drivers/net/mlx5/mlx5_rxq.c | 41 +-- > drivers/net/mlx5/mlx5_socket.c | 4 +- > drivers/net/mlx5/mlx5_stats.c | 22 +- > drivers/net/mlx5/mlx5_trigger.c | 2 +- > drivers/net/mlx5/mlx5_txq.c | 24 +- > 15 files changed, 879 insertions(+), 338 deletions(-) >=20 > -- > 1.8.3.1