From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 987A7A0562 for ; Sun, 1 Mar 2020 16:31:37 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 035D41C066; Sun, 1 Mar 2020 16:31:37 +0100 (CET) Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by dpdk.org (Postfix) with ESMTP id C84CD1C05C for ; Sun, 1 Mar 2020 16:31:35 +0100 (CET) Received: by mail-wr1-f68.google.com with SMTP id n7so1406700wrt.11 for ; Sun, 01 Mar 2020 07:31:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=y9B97EHnNEA+USK7s6xDZcscWppuCwGTxrQHKo8oYII=; b=pyh+Y0sKaZyKRMuxcHMWRIClGxMOwIf4hvWSc3xmrl+BCgPrZYoY669CkIsqq54fMO L9y8Ehk/30WtvsEZbx94xcTBXdhXHZmkrRtjw7HDxHwUggmJ7LuzeI3HPnXvYfG2OVJY 2Be8wxh7+JuKZVwenoKKkgIBcpdy44kRWlQipykU6B7y80Hz2SfVqY5+Cj8Deh1Bisrn 2t3cyw9eLFyYyc+XBGXfp1FHsH7QtouwndU2tA9RqMPkM3H3KSPix9z21W6PQ+d8v+VA MMJSwdJ6Dud9j86UxasrN3fDEpIPvehTEicXb/qzobjZx+evlnG470IWS/gOZB+fwbQl UnOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=y9B97EHnNEA+USK7s6xDZcscWppuCwGTxrQHKo8oYII=; b=KbX+/MXgmJqTXhWy/mXCYqj35Q3SHQ3tZyCcqdb+WZJtDOaoP/keBrD6PEqCwn7I8l CC8/4VvE44cwF6Hg7gPwl+dK46NdcYPrj2DMll/OULlYYGad7d1h9y5BMp2qpcqgCzha AdBZHvbA5oVjgvCPR38RhBDU1y6HU25IT9RIXCZFGxLPXCXf6a23xHAG1Y/ZPjN6t4Nh hp7wd9yOq/AuNZ7WsedmOX+5pGxUgtrXRSx5mh0NSFVNDpI5kLkHDrArDKJxDmOD08jw ve3LhcjCFP2GYeVuAfjRc8BXBF7hIWk3OIM3rHu4tC5vCtCnBLItDs2HteqZ9wtPKLjL D7Lw== X-Gm-Message-State: APjAAAX8KqRK/+DR1XIT5q2hxhvJWgRAMjenNtVybrn32DQN+cAcfdiI 7ieN3SdZtuhpEfID/3K58F8DMuykRK/Anpf4toE= X-Google-Smtp-Source: APXvYqzILFoJax1HF6a1EZX6aKAHKt+6u8X+9ZWZ2ruNl+RvRi/948e165aPcVp3PNwAzNt4aPlImRCz4fwMJAr1Whc= X-Received: by 2002:a5d:6b03:: with SMTP id v3mr16152004wrw.289.1583076695386; Sun, 01 Mar 2020 07:31:35 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Cliff Burdick Date: Sun, 1 Mar 2020 07:31:24 -0800 Message-ID: To: "Xueming(Steven) Li" Cc: "users@dpdk.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] RDMA over DPDK X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" If you're interested in this for GPUs, you should check out cuVNF here: https://developer.nvidia.com/aerial-sdk On Sun, Mar 1, 2020 at 3:33 AM Xueming(Steven) Li wrote: > With a quick hack on mlx5 pmd, it's possible to send RDMA operation with > few changes. Performance result between 25Gb back to back connected NICs: > > - Continues 1MB RDMA write on 256 different memory target of remote > peer: line speed, 2.6Mpps, MTU 1024 > - Continues 8B RDMA write to remote peer: line speed, 29.4Mpps, > RoCE2(74B+8B) > > Currently, dpdk usage focus on network scenario: ovs, firewall, load > balance... > With hw acceleration, RDMA is application level api with more capability > than sockets, 2GB xfer, less latency and atomic operations, it will enable > dpdk bypass stack to application server - another huge market I believe. > > Why RDMA over dpdk: > > - performance , dpdk style batch/burst xfer, less i-cache miss > - easy to prefetch - no linked list > - reuse mbuf data structure with some modification > - able to send rdma request with eth mbuf data > - virtualization support, with rte_flow, able to do hw encap/decap for VF > RDMA traffic > > Potential application: > > - rdma <-> rdma application in DC/HPC > - eth <-> rdma application > - device power saving. If pc/mobile support rdma, playing video or > download file, most networking xfer happens with few cpu involvement. > > Interested? > > Xueming Li >