From: Maxime Coquelin <maxime.coquelin@redhat.com>
To: dev@dpdk.org, chenbo.xia@intel.com, david.marchand@redhat.com,
eperezma@redhat.com, stephen@networkplumber.org
Subject: Re: [PATCH 00/21] Add control queue & MQ support to Virtio-user vDPA
Date: Thu, 9 Feb 2023 10:21:34 +0100 [thread overview]
Message-ID: <371044fc-dd93-d77d-8e5d-1ea557ff2782@redhat.com> (raw)
In-Reply-To: <20230209091710.485512-1-maxime.coquelin@redhat.com>
Oups, forgot to set subject prefix to v3.
On 2/9/23 10:16, Maxime Coquelin wrote:
> This series introduces control queue support for Vhost-vDPA
> backend. This is a requirement to support multiqueue, but
> be usefull for other features like RSS for example.
>
> Since the Virtio-user layer of the Virtio PMD must handle
> some control messages, like the number of queue pairs to
> be used by the device, a shadow control queue is created
> at Virtio-user layer.
>
> Control messages from the regular Virtio control queue
> are still dequeues and handled if needed by the Virtio-user
> layer, and are then forwarded to the shadow control queue
> so that the physical vDPA device can handle them.
>
> This model is similar to the one adopted by the QEMU
> project.
>
> In order to avoid code duplication, virtqueue allocation
> and control queue message sending has been factored out
> of the Virtio layer to be reusable by the Virtio-user
> layer.
>
> Finally, in order to support vDPA hardware which may
> support large number of queues, last patch removes the
> 8 queue pairs limitation by dynamically allocating
> vring metadata.
>
> The series has been tested with Nvidia Cx-6 DX NIC
> with up to 16 queue pairs:
>
> # echo 0 > /sys/bus/pci/devices/0000\:3b\:00.0/sriov_numvfs
> # echo 0 > /sys/bus/pci/devices/0000\:3b\:00.1/sriov_numvfs
> # modprobe vhost_vdpa
> # modprobe mlx5_vdpa
> # echo 1 > /sys/bus/pci/devices/0000\:3b\:00.0/sriov_numvfs
> # echo 0000:3b:00.2 >/sys/bus/pci/drivers/mlx5_core/unbind
> # devlink dev eswitch set pci/0000:3b:00.0 mode switchdev
> # echo 0000:3b:00.2 >/sys/bus/pci/drivers/mlx5_core/bind
> # vdpa dev add name vdpa0 mgmtdev pci/0000:3b:00.2 mac 00:11:22:33:44:03 max_vqp 16
> # ulimit -l unlimited
> # dpdk-testpmd -l 0,2,4,6 --socket-mem 1024,0 --vdev 'virtio_user0,path=/dev/vhost-vdpa-0' --no-pci -n 3 -- --nb-cores=3 -i --rxq=16 --txq=16
>
> Changes in v3:
> ==============
> - Trivial code simplifications (Eugenio)
>
> Changes in v2:
> ==============
> - Fix double spaces (Chenbo)
> - Get rid of uneeded gotos (Stephen)
> - Only allocate packed ring metadata if supported (Chenbo)
> - Rebased on top of main
>
> Maxime Coquelin (21):
> net/virtio: move CVQ code into a dedicated file
> net/virtio: introduce notify callback for control queue
> net/virtio: virtqueue headers alloc refactoring
> net/virtio: remove port ID info from Rx queue
> net/virtio: remove unused fields in Tx queue struct
> net/virtio: remove unused queue ID field in Rx queue
> net/virtio: remove unused Port ID in control queue
> net/virtio: move vring memzone to virtqueue struct
> net/virtio: refactor indirect desc headers init
> net/virtio: alloc Rx SW ring only if vectorized path
> net/virtio: extract virtqueue init from virtio queue init
> net/virtio-user: fix device starting failure handling
> net/virtio-user: simplify queues setup
> net/virtio-user: use proper type for number of queue pairs
> net/virtio-user: get max number of queue pairs from device
> net/virtio-user: allocate shadow control queue
> net/virtio-user: send shadow virtqueue info to the backend
> net/virtio-user: add new callback to enable control queue
> net/virtio-user: forward control messages to shadow queue
> net/virtio-user: advertize control VQ support with vDPA
> net/virtio-user: remove max queues limitation
>
> drivers/net/virtio/meson.build | 1 +
> drivers/net/virtio/virtio.h | 6 -
> drivers/net/virtio/virtio_cvq.c | 229 +++++++++
> drivers/net/virtio/virtio_cvq.h | 127 +++++
> drivers/net/virtio/virtio_ethdev.c | 472 +-----------------
> drivers/net/virtio/virtio_rxtx.c | 47 +-
> drivers/net/virtio/virtio_rxtx.h | 31 +-
> drivers/net/virtio/virtio_rxtx_packed.c | 3 +-
> drivers/net/virtio/virtio_rxtx_simple.c | 3 +-
> drivers/net/virtio/virtio_rxtx_simple.h | 7 +-
> .../net/virtio/virtio_rxtx_simple_altivec.c | 4 +-
> drivers/net/virtio/virtio_rxtx_simple_neon.c | 4 +-
> drivers/net/virtio/virtio_rxtx_simple_sse.c | 4 +-
> drivers/net/virtio/virtio_user/vhost.h | 1 +
> drivers/net/virtio/virtio_user/vhost_vdpa.c | 16 +-
> .../net/virtio/virtio_user/virtio_user_dev.c | 305 +++++++++--
> .../net/virtio/virtio_user/virtio_user_dev.h | 30 +-
> drivers/net/virtio/virtio_user_ethdev.c | 49 +-
> drivers/net/virtio/virtqueue.c | 346 ++++++++++++-
> drivers/net/virtio/virtqueue.h | 127 +----
> 20 files changed, 1066 insertions(+), 746 deletions(-)
> create mode 100644 drivers/net/virtio/virtio_cvq.c
> create mode 100644 drivers/net/virtio/virtio_cvq.h
>
next prev parent reply other threads:[~2023-02-09 9:21 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-02-09 9:16 Maxime Coquelin
2023-02-09 9:16 ` [PATCH 01/21] net/virtio: move CVQ code into a dedicated file Maxime Coquelin
2023-02-09 9:16 ` [PATCH 02/21] net/virtio: introduce notify callback for control queue Maxime Coquelin
2023-02-09 9:16 ` [PATCH 03/21] net/virtio: virtqueue headers alloc refactoring Maxime Coquelin
2023-02-09 9:16 ` [PATCH 04/21] net/virtio: remove port ID info from Rx queue Maxime Coquelin
2023-02-09 9:16 ` [PATCH 05/21] net/virtio: remove unused fields in Tx queue struct Maxime Coquelin
2023-02-09 9:16 ` [PATCH 06/21] net/virtio: remove unused queue ID field in Rx queue Maxime Coquelin
2023-02-09 9:16 ` [PATCH 07/21] net/virtio: remove unused Port ID in control queue Maxime Coquelin
2023-02-09 9:16 ` [PATCH 08/21] net/virtio: move vring memzone to virtqueue struct Maxime Coquelin
2023-02-09 9:16 ` [PATCH 09/21] net/virtio: refactor indirect desc headers init Maxime Coquelin
2023-02-09 9:16 ` [PATCH 10/21] net/virtio: alloc Rx SW ring only if vectorized path Maxime Coquelin
2023-02-09 9:17 ` [PATCH 11/21] net/virtio: extract virtqueue init from virtio queue init Maxime Coquelin
2023-02-09 9:17 ` [PATCH 12/21] net/virtio-user: fix device starting failure handling Maxime Coquelin
2023-02-09 9:17 ` [PATCH 13/21] net/virtio-user: simplify queues setup Maxime Coquelin
2023-02-09 9:17 ` [PATCH 14/21] net/virtio-user: use proper type for number of queue pairs Maxime Coquelin
2023-02-09 9:17 ` [PATCH 15/21] net/virtio-user: get max number of queue pairs from device Maxime Coquelin
2023-02-09 9:17 ` [PATCH 16/21] net/virtio-user: allocate shadow control queue Maxime Coquelin
2023-02-09 9:17 ` [PATCH 17/21] net/virtio-user: send shadow virtqueue info to the backend Maxime Coquelin
2023-02-09 9:17 ` [PATCH 18/21] net/virtio-user: add new callback to enable control queue Maxime Coquelin
2023-02-09 9:17 ` [PATCH 19/21] net/virtio-user: forward control messages to shadow queue Maxime Coquelin
2023-02-09 9:17 ` [PATCH 20/21] net/virtio-user: advertize control VQ support with vDPA Maxime Coquelin
2023-02-09 9:17 ` [PATCH 21/21] net/virtio-user: remove max queues limitation Maxime Coquelin
2023-02-09 9:21 ` Maxime Coquelin [this message]
2023-02-09 12:12 ` [PATCH 00/21] Add control queue & MQ support to Virtio-user vDPA Maxime Coquelin
-- strict thread matches above, loose matches on Subject: below --
2023-02-07 15:15 Maxime Coquelin
2023-02-07 15:17 ` Maxime Coquelin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=371044fc-dd93-d77d-8e5d-1ea557ff2782@redhat.com \
--to=maxime.coquelin@redhat.com \
--cc=chenbo.xia@intel.com \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=eperezma@redhat.com \
--cc=stephen@networkplumber.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).