From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6B2DEA052E for ; Sun, 1 Mar 2020 12:33:18 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A4F901C02D; Sun, 1 Mar 2020 12:33:17 +0100 (CET) Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20086.outbound.protection.outlook.com [40.107.2.86]) by dpdk.org (Postfix) with ESMTP id 4B8221C02B for ; Sun, 1 Mar 2020 12:33:16 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KNr2P2br3NKi1Vcg4q3gU4BFTF4ixt8f98UI+ChNkWqB1MtxIiCxfy7T8qwZS2iBCqsAebUfi39QmDTunko/JTMmW2T2cFPAUo8VDoCI5w3L3fiITCXPjG/w3RUg62cgen9GqLTfUkgq8gxxky33rY/CX1IykXkLsmfxUuksfoOLcjC2ZUETUx5yI+xKamlizhdFooj5ZxjA83+KGCVQ9Llj6kHppI7/m8gusVG3m23GXTmXhLfpDBZO0Xpl909o5Q6JMJQUlkldyGRB/rJRdBdP/+LSikTsTmpAlaEjKWvltWF2v04qnBpipFTIoMgiy4CuhwEJEDwzAFTUNcEwVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OEwjvveCwU6DF29Usq59lg+EMfUVf+E6QZUSOh3jNf4=; b=R4CHwWtXxYJEwbL935vEt0Ebvmi4vU1os/RanZTxEKPWlE/p5E3J6UNbuTvFz86AAwSPddddFnxTsNBTMf3aP0BR0J+qZW6RDFIbPyB2ai3GQk5tDr7wwqV7kmvgPL2+qfh0YPfd4xHTRHqbQeqmG3ys9QS/paDWJiBSDRu7ILlX5xDM+U7U8MYQD/ekrQaj7dDlcMMCAbz026kqazix0SzXuE/8SAyJeGRPnhtmqIjDEDQbdB/WFtIqg+Q6nSeEN/ZG/EQ3Dv7FXxbMwNfk24qtOxLlq9WC/UXi7FAuUvT0h0dZ6Lg0MEvF0tVPFOYz1suJ6mOC83Zw04ItUzp6MQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OEwjvveCwU6DF29Usq59lg+EMfUVf+E6QZUSOh3jNf4=; b=NKqKnCPq4xFLGOlaIGZRcd0GB2iAEqlY6837r0z/hSM9mAHxMnwWBOByNdED6cuKZFLu5AVGcLVo9Flq+0t5qavQccfalKJQAM//ZIWT/U6CIywY4iE+UnaLxVX4ERW4CHvqMl46H5ai/6ugZdHqgs027BLGuq07QBOYTCx56+Q= Received: from VI1PR05MB3197.eurprd05.prod.outlook.com (10.170.239.28) by VI1PR05MB4784.eurprd05.prod.outlook.com (20.176.7.152) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2772.18; Sun, 1 Mar 2020 11:33:04 +0000 Received: from VI1PR05MB3197.eurprd05.prod.outlook.com ([fe80::ad75:6f18:939b:a3a5]) by VI1PR05MB3197.eurprd05.prod.outlook.com ([fe80::ad75:6f18:939b:a3a5%6]) with mapi id 15.20.2772.018; Sun, 1 Mar 2020 11:33:04 +0000 From: "Xueming(Steven) Li" To: "users@dpdk.org" Thread-Topic: RDMA over DPDK Thread-Index: AdXvvF3EaXyXeMJaS8S/PbcD1DZ+iQ== Date: Sun, 1 Mar 2020 11:33:04 +0000 Message-ID: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=xuemingl@mellanox.com; x-originating-ip: [240e:e0:cf28:1600:a44c:6970:6f3a:358a] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: a4a2b904-52ad-4df4-d5ea-08d7bdd44e9f x-ms-traffictypediagnostic: VI1PR05MB4784: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3631; x-forefront-prvs: 0329B15C8A x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(376002)(136003)(346002)(396003)(39860400002)(366004)(199004)(189003)(7696005)(6506007)(6916009)(86362001)(3480700007)(33656002)(5660300002)(478600001)(52536014)(64756008)(8936002)(81166006)(81156014)(8676002)(71200400001)(186003)(9686003)(316002)(55016002)(2906002)(76116006)(7116003)(66446008)(66556008)(66476007)(66946007); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR05MB4784; H:VI1PR05MB3197.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ALv2xDJQzUPLmmWEMJkXh0TnaKWf5rXz3YyzVB00RqLNkD9wQ1p4SMUoVKt/2VuRmO84TjK1vnP+z3IRmeB/OyJwVKvZTudwrdVez1rLGjBM7jKUwr+WFB6Xl7u3DTof0rz5MPUkkD/eQ5klcNGzhk+EBtUrMaeT17dZVS5UvWu50IZMggvR8DmJmu84v7PWrcCaJ2d+PzsgA+rQ9t0mkA7YY4pg8p0dH2LQEWQ13cUdM4bQ6uckXjUeLZq/vSzzjDuTTPPygibpHbxnn3nDyO1P0sC2TlAL6z/CJqNoRoXNqkPm8QLBJksMK1BRK0d3Ud84sEKxSVVgMZzqVC+CDiP08WofM2KLPEfD20dFtiWZfM9sqxfHQ7d3hrD1zrJmKn++AxiUf7Hut3/D6pnqB1Jerj838wy+lPafLQhXKcFdi6xXS0SySVimLabToSnT x-ms-exchange-antispam-messagedata: 2UezH8QbnZzID1ssxB0LkJOtZdYzv9e0bm5DJ/y0kWSm977j5acTAUIDCHPyTH2tIgJMk6QamlJ2xNwBVQBrqs4u+IYoVj1IYOMjtnbCKtOCorWoTN1hJrXPKw9sbmm+qyHVyk85OzZEMdVhMp9vBs0Vk05SA8hlkEhSNz3ey+dwbUDWuTYM6JK3Fcjv3OmvXTdJ/2nBHEdvl0jENy5qiw== x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: a4a2b904-52ad-4df4-d5ea-08d7bdd44e9f X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Mar 2020 11:33:04.3560 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: hjcsT/Twv09Qp1mTChzMLpvuqjvWxyV21olmbGQ/TCwYHEpZGOmYKLTse3sviAwlw1vHkhvky0VJMvSJBEbnEA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR05MB4784 Subject: [dpdk-users] RDMA over DPDK X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" =A0With a quick hack on mlx5 pmd, it's possible to send RDMA operation with= few changes. Performance result between 25Gb back to back connected NICs: - Continues 1MB RDMA write on 256 different memory target of remote pee= r: line speed, 2.6Mpps, MTU 1024 - Continues 8B RDMA write to remote peer: line speed, 29.4Mpps, RoCE2(7= 4B+8B)=20 Currently, dpdk usage focus on network scenario: ovs, firewall, load balanc= e... With hw acceleration, RDMA is application level api with more capability th= an sockets, 2GB xfer, less latency and atomic operations, it will enable dp= dk bypass stack to application server - another huge market I believe. Why RDMA over dpdk: - performance , dpdk style batch/burst xfer, less i-cache miss - easy to prefetch - no linked list - reuse mbuf data structure with some modification - able to send rdma request with eth mbuf data - virtualization support, with rte_flow, able to do hw encap/decap for VF R= DMA traffic =A0 Potential application: - rdma <-> rdma application in DC/HPC - eth <-> rdma application - device power saving. If pc/mobile support rdma, playing video or download= file, most networking xfer happens with few cpu involvement. Interested? Xueming Li