From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A5D6342C5E; Thu, 8 Jun 2023 14:44:49 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 31E55406B5; Thu, 8 Jun 2023 14:44:49 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 0765840042 for ; Thu, 8 Jun 2023 14:44:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686228286; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wOPkVl1kyw8IhxC8mjeAn/wDr81+V1VSvUzc9z5kgfI=; b=I3c/XtKyX+yegQiJyEQhVClBKlJ6iW2ScaY1DA0jEhhnuYQqIQOQlQelUk0TROaxHX1V7Q ecRhKFxqj9G2ccwLt2D0kS5SGgsnkPLbo5OE7NJTzl7Z4sIt5BhE/ru5aDk7CTXHXNP6sV +z8dGPA005HGzyVWSrzHeG5mahaTW0w= Received: from mail-oi1-f200.google.com (mail-oi1-f200.google.com [209.85.167.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-649-kIXWTd7FPW2780NMwZTdIg-1; Thu, 08 Jun 2023 08:44:45 -0400 X-MC-Unique: kIXWTd7FPW2780NMwZTdIg-1 Received: by mail-oi1-f200.google.com with SMTP id 5614622812f47-3922d2460afso576927b6e.3 for ; Thu, 08 Jun 2023 05:44:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686228284; x=1688820284; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wOPkVl1kyw8IhxC8mjeAn/wDr81+V1VSvUzc9z5kgfI=; b=IzyIFnNYBjvjsjF8cOAeqaAP6vqhIQX+DqdZ1MRgzpg4LTyEjaBdSnS/xptm6/exhz bU+ghcveEMQ85V10ST6dt4uQmDxn6mzoA0xAl1O8vB1D676NCsF7MAbOoUBpR+NXzYIY 1Ivn/KCxnF1J6GQcwY/tSHUb47S7I3mn00sY9TTC/H8jRXt6SwT+f7OpxyfAkU4p89bb L+fMPOaoHWMVUCosooIsEZic08DMqxp1GQd3gWT2bTzhA+5Y9bHYZ3I5unfuD8j2GObo TKFFmhzVcVKhWWT7Ia4qs6zMfzXSPZM5zGaFDrYZIHLSk411vdOICM/vU+vJt1Ht9KAc vrhg== X-Gm-Message-State: AC+VfDwmivN3RD5h8rIn6lgbp2EY9wCbH2p0kzbqWz+R/XQ4IXqyHbEf OLpAPKiwT+fT7kHBcEuh4n6x019stkCE3O5LDQv7/Chk2/Dui3q/lQqk2B1fuABuCpYpuz852gY GbQtFDxpspR+/8c4ihJc= X-Received: by 2002:a05:6808:a8a:b0:383:ca99:c70e with SMTP id q10-20020a0568080a8a00b00383ca99c70emr7781003oij.59.1686228284177; Thu, 08 Jun 2023 05:44:44 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4g9c8ro1lptJ31eHzBjRiacpOIRLlNyOFRvRYUuclMCngDF+GpczLj3FIp7QhTVFEtwc2foD/hk41+zgnuBMA= X-Received: by 2002:a05:6808:a8a:b0:383:ca99:c70e with SMTP id q10-20020a0568080a8a00b00383ca99c70emr7780987oij.59.1686228283855; Thu, 08 Jun 2023 05:44:43 -0700 (PDT) MIME-Version: 1.0 References: <20230606081852.71003-1-maxime.coquelin@redhat.com> <088c44c5-0b01-f519-dff7-004f50d68573@redhat.com> In-Reply-To: <088c44c5-0b01-f519-dff7-004f50d68573@redhat.com> From: David Marchand Date: Thu, 8 Jun 2023 14:44:32 +0200 Message-ID: Subject: Re: [PATCH v5 00/26] Add VDUSE support to Vhost library To: Maxime Coquelin Cc: dev@dpdk.org, chenbo.xia@intel.com, mkp@redhat.com, fbl@redhat.com, jasowang@redhat.com, cunming.liang@intel.com, xieyongji@bytedance.com, echaudro@redhat.com, eperezma@redhat.com, amorenoz@redhat.com, lulu@redhat.com, Thomas Monjalon X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Thu, Jun 8, 2023 at 11:17=E2=80=AFAM Maxime Coquelin wrote: > On 6/7/23 10:05, David Marchand wrote: > > On Tue, Jun 6, 2023 at 10:19=E2=80=AFAM Maxime Coquelin > > wrote: > >> > >> This series introduces a new type of backend, VDUSE, > >> to the Vhost library. > >> > >> VDUSE stands for vDPA device in Userspace, it enables > >> implementing a Virtio device in userspace and have it > >> attached to the Kernel vDPA bus. > >> > >> Once attached to the vDPA bus, it can be used either by > >> Kernel Virtio drivers, like virtio-net in our case, via > >> the virtio-vdpa driver. Doing that, the device is visible > >> to the Kernel networking stack and is exposed to userspace > >> as a regular netdev. > >> > >> It can also be exposed to userspace thanks to the > >> vhost-vdpa driver, via a vhost-vdpa chardev that can be > >> passed to QEMU or Virtio-user PMD. > >> > >> While VDUSE support is already available in upstream > >> Kernel, a couple of patches are required to support > >> network device type: > >> > >> https://gitlab.com/mcoquelin/linux/-/tree/vduse_networking_rfc > >> > >> In order to attach the created VDUSE device to the vDPA > >> bus, a recent iproute2 version containing the vdpa tool is > >> required. > >> > >> Benchmark results: > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > >> > >> On this v2, PVP reference benchmark has been run & compared with > >> Vhost-user. > >> > >> When doing macswap forwarding in the worload, no difference is seen. > >> When doing io forwarding in the workload, we see 4% performance > >> degradation with VDUSE, comapred to Vhost-user/Virtio-user. It is > >> explained by the use of the IOTLB layer in the Vhost-library when usin= g > >> VDUSE, whereas Vhost-user/Virtio-user does not make use of it. > >> > >> Usage: > >> =3D=3D=3D=3D=3D=3D > >> > >> 1. Probe required Kernel modules > >> # modprobe vdpa > >> # modprobe vduse > >> # modprobe virtio-vdpa > >> > >> 2. Build (require vduse kernel headers to be available) > >> # meson build > >> # ninja -C build > >> > >> 3. Create a VDUSE device (vduse0) using Vhost PMD with > >> testpmd (with 4 queue pairs in this example) > >> # ./build/app/dpdk-testpmd --no-pci --vdev=3Dnet_vhost0,iface=3D/dev/v= duse/vduse0,queues=3D4 --log-level=3D*:9 -- -i --txq=3D4 --rxq=3D4 > > > > 9 is a nice but undefined value. 8 is enough. > > In general, I prefer "human readable" strings, like *:debug ;-). > > > > > >> > >> 4. Attach the VDUSE device to the vDPA bus > >> # vdpa dev add name vduse0 mgmtdev vduse > >> =3D> The virtio-net netdev shows up (eth0 here) > >> # ip l show eth0 > >> 21: eth0: mtu 1500 qdisc mq state UP= mode DEFAULT group default qlen 1000 > >> link/ether c2:73:ea:a7:68:6d brd ff:ff:ff:ff:ff:ff > >> > >> 5. Start/stop traffic in testpmd > >> testpmd> start > >> testpmd> show port stats 0 > >> ######################## NIC statistics for port 0 ###############= ######### > >> RX-packets: 11 RX-missed: 0 RX-bytes: 1482 > >> RX-errors: 0 > >> RX-nombuf: 0 > >> TX-packets: 1 TX-errors: 0 TX-bytes: 62 > >> > >> Throughput (since last show) > >> Rx-pps: 0 Rx-bps: 0 > >> Tx-pps: 0 Tx-bps: 0 > >> ###################################################################= ######### > >> testpmd> stop > >> > >> 6. Detach the VDUSE device from the vDPA bus > >> # vdpa dev del vduse0 > >> > >> 7. Quit testpmd > >> testpmd> quit > >> > >> Known issues & remaining work: > >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D > >> - Fix issue in FD manager (still polling while FD has been removed) > >> - Add Netlink support in Vhost library > >> - Support device reconnection > >> -> a temporary patch to support reconnection via a tmpfs file is ava= ilable, > >> upstream solution would be in-kernel and is being developed. > >> -> https://gitlab.com/mcoquelin/dpdk-next-virtio/-/commit/5ad06ce141= 59a9ce36ee168dd13ef389cec91137 > >> - Support packed ring > >> - Provide more performance benchmark results > > > > We are missing a reference to the kernel patches required to have > > vduse accept net devices. > > Right, I mention it in the cover letter, but it should be in the release > note also. I propose to append this to the release note: > "While VDUSE support is already available in upstream Kernel, a couple > of patches are required to support network device type, which are being > upstreamed: > https://lore.kernel.org/all/20230419134329.346825-1-maxime.coquelin@redha= t.com/" > > Does that sound good to you? Ok for me. Thanks. --=20 David Marchand