From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0960E42923; Wed, 12 Apr 2023 08:57:19 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0190242C24; Wed, 12 Apr 2023 08:56:58 +0200 (CEST) Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) by mails.dpdk.org (Postfix) with ESMTP id C726740DF6 for ; Thu, 6 Apr 2023 05:45:11 +0200 (CEST) Received: by mail-pf1-f179.google.com with SMTP id u38so24957734pfg.10 for ; Wed, 05 Apr 2023 20:45:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1680752710; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=a6LWNLspSEXd/soMk9mvpXw9vYvnp8JFWiJz6PDZx50=; b=I1CXsQ7MXA2DiwTswLCWVhsqMOjob7rFHAM/n78oOu/wy/8UqxYo9a9OVYPlkgkVjX EnE3Qyli8fI+splmeFOhJpnLh+kdTTVBDioOTcRBkT4wxsc8UxzFm4U+DDtcTn/cw1Gn 5xupXXraTIXu/p+ZQjSPRJ3XVU2SAg7G2T/HSrLUppWfP+CBoMLTAZqwMH7WCyKurDWM tNnG/s/KkQ/xu0L4Xtmswc4UvkROnYD2jrXTzSJBtlYkZnLgFtTkLCIneWmH2pmx361P 8RlV4WfD0UZJ6MKJmsMx1Mzolaeoaa9apP5v3066geeqdpkGY9VnR039iACsoeYqNVUa /NWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680752710; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=a6LWNLspSEXd/soMk9mvpXw9vYvnp8JFWiJz6PDZx50=; b=ud8K6wQ9Fn8HlVIW36mLY7+/hqicezSQ/HoBZRjFRSRu379YmzKoSnNdVVSBbatEEh DA38TN163Z99Zb3nzY3umsmegYUvpcWY+xm9bk47jFaxemSlhQMY5OoZ/Pyj3dR/T9yQ IG7UqloIW5jIrdwBTT36byNavtGb+hZMcpY8ncCTE/+h4NTJ4gy4Q5xMUMaqpLBh9zCp TN52xt3DvBix5OEtgKgQOH+DKhE/6TPudormBEtx3Oj0Qmdqsj8JaIkOA05WV3lvvp7I I5WKyhsVb/JcrLAcqNNqGdA4O+aJKbLHit6XE5DIaMnqt6FqrKAEbaxlhpyCexFeb8xu Wstw== X-Gm-Message-State: AAQBX9d0Enpu5jq5tHPKqhozya7H1sxMAYFhGC2gab9q7BwiMvHtVbZr deY/9uoi/uLlsmal1TYnwyYPN9w9vW8Ypf1R48ZP X-Google-Smtp-Source: AKy350a057OYz3tahpoxMhb+Kk2l/Bq8vBiB+V56DhZ3fd07z8yFlLI08rD22n/w3l/ISsPzevBFFMMjffw5DDtYKw4= X-Received: by 2002:a63:ed0f:0:b0:513:9238:8c16 with SMTP id d15-20020a63ed0f000000b0051392388c16mr2769964pgi.0.1680752710465; Wed, 05 Apr 2023 20:45:10 -0700 (PDT) MIME-Version: 1.0 References: <20230331154259.1447831-1-maxime.coquelin@redhat.com> In-Reply-To: <20230331154259.1447831-1-maxime.coquelin@redhat.com> From: Yongji Xie Date: Thu, 6 Apr 2023 11:44:59 +0800 Message-ID: Subject: Re: [RFC 00/27] Add VDUSE support to Vhost library To: Maxime Coquelin Cc: dev@dpdk.org, David Marchand , chenbo.xia@intel.com, mkp@redhat.com, fbl@redhat.com, Jason Wang , cunming.liang@intel.com, echaudro@redhat.com, Eugenio Perez Martin , Adrian Moreno Zapata Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Wed, 12 Apr 2023 08:56:51 +0200 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Maxime, On Fri, Mar 31, 2023 at 11:43=E2=80=AFPM Maxime Coquelin wrote: > > This series introduces a new type of backend, VDUSE, > to the Vhost library. > > VDUSE stands for vDPA device in Userspace, it enables > implementing a Virtio device in userspace and have it > attached to the Kernel vDPA bus. > > Once attached to the vDPA bus, it can be used either by > Kernel Virtio drivers, like virtio-net in our case, via > the virtio-vdpa driver. Doing that, the device is visible > to the Kernel networking stack and is exposed to userspace > as a regular netdev. > > It can also be exposed to userspace thanks to the > vhost-vdpa driver, via a vhost-vdpa chardev that can be > passed to QEMU or Virtio-user PMD. > > While VDUSE support is already available in upstream > Kernel, a couple of patches are required to support > network device type: > > https://gitlab.com/mcoquelin/linux/-/tree/vduse_networking_poc > > In order to attach the created VDUSE device to the vDPA > bus, a recent iproute2 version containing the vdpa tool is > required. > > Usage: > =3D=3D=3D=3D=3D=3D > > 1. Probe required Kernel modules > # modprobe vdpa > # modprobe vduse > # modprobe virtio-vdpa > > 2. Build (require vduse kernel headers to be available) > # meson build > # ninja -C build > > 3. Create a VDUSE device (vduse0) using Vhost PMD with > testpmd (with 4 queue pairs in this example) > # ./build/app/dpdk-testpmd --no-pci --vdev=3Dnet_vhost0,iface=3D/dev/vdus= e/vduse0,queues=3D4 --log-level=3D*:9 -- -i --txq=3D4 --rxq=3D4 > > 4. Attach the VDUSE device to the vDPA bus > # vdpa dev add name vduse0 mgmtdev vduse > =3D> The virtio-net netdev shows up (eth0 here) > # ip l show eth0 > 21: eth0: mtu 1500 qdisc mq state UP mo= de DEFAULT group default qlen 1000 > link/ether c2:73:ea:a7:68:6d brd ff:ff:ff:ff:ff:ff > > 5. Start/stop traffic in testpmd > testpmd> start > testpmd> show port stats 0 > ######################## NIC statistics for port 0 ###################= ##### > RX-packets: 11 RX-missed: 0 RX-bytes: 1482 > RX-errors: 0 > RX-nombuf: 0 > TX-packets: 1 TX-errors: 0 TX-bytes: 62 > > Throughput (since last show) > Rx-pps: 0 Rx-bps: 0 > Tx-pps: 0 Tx-bps: 0 > #######################################################################= ##### > testpmd> stop > > 6. Detach the VDUSE device from the vDPA bus > # vdpa dev del vduse0 > > 7. Quit testpmd > testpmd> quit > > Known issues & remaining work: > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D > - Fix issue in FD manager (still polling while FD has been removed) > - Add Netlink support in Vhost library > - Support device reconnection > - Support packed ring > - Enable & test more Virtio features > - Provide performance benchmark results > Nice work! Thanks for bringing VDUSE to the network area. I wonder if you have some plan to support userspace memory registration [1]? I think this feature can benefit the performance since an extra data copy could be eliminated in our case. [1] https://lwn.net/Articles/902809/ Thanks, Yongji