From: "Xueming(Steven) Li" <xuemingl@nvidia.com>
To: "maxime.coquelin@redhat.com" <maxime.coquelin@redhat.com>,
"dev@dpdk.org" <dev@dpdk.org>
Cc: Matan Azrad <matan@nvidia.com>, Slava Ovsiienko <viacheslavo@nvidia.com>
Subject: Re: [dpdk-dev] [PATCH 2/2] vdpa/mlx5: retry VAR allocation during vDPA restart
Date: Wed, 13 Oct 2021 10:14:13 +0000 [thread overview]
Message-ID: <de4de5021946242f89b4d742d9b85574a63fc79b.camel@nvidia.com> (raw)
In-Reply-To: <033827d7-8f70-80e0-7dbb-22f501bc6df2@redhat.com>
On Wed, 2021-10-13 at 12:06 +0200, Maxime Coquelin wrote:
>
> On 9/23/21 10:17, Xueming Li wrote:
> > VAR is the device memory space for the virtio queues doorbells, qemu
> > could mmap it to directly to speed up doorbell push.
> >
> > On a busy system, Qemu takes time to release VAR resources during driver
> > shutdown. If vdpa restarted quickly, the VAR allocation failed with
> > error 28 since the VAR is singleton resource per device.
> >
> > This patch adds retry mechanism for VAR allocation.
> >
> > Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> > Reviewed-by: Matan Azrad <matan@nvidia.com>
> > ---
> > drivers/vdpa/mlx5/mlx5_vdpa.c | 9 ++++++++-
> > 1 file changed, 8 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.c b/drivers/vdpa/mlx5/mlx5_vdpa.c
> > index 6d17d7a6f3..991739e984 100644
> > --- a/drivers/vdpa/mlx5/mlx5_vdpa.c
> > +++ b/drivers/vdpa/mlx5/mlx5_vdpa.c
> > @@ -693,7 +693,14 @@ mlx5_vdpa_dev_probe(struct rte_device *dev)
> > if (attr.num_lag_ports == 0)
> > priv->num_lag_ports = 1;
> > priv->ctx = ctx;
> > - priv->var = mlx5_glue->dv_alloc_var(ctx, 0);
> > + for (retry = 0; retry < 7; retry++) {
> > + priv->var = mlx5_glue->dv_alloc_var(ctx, 0);
> > + if (priv->var != NULL)
> > + break;
> > + DRV_LOG(WARNING, "Failed to allocate VAR, retry %d.\n", retry);
> > + /* Wait Qemu release VAR during vdpa restart, 0.1 sec based. */
> > + usleep(100000U << retry);
> > + }
> > if (!priv->var) {
> > DRV_LOG(ERR, "Failed to allocate VAR %u.", errno);
> > goto error;
> >
>
> That looks fragile, but at least we have a warning we can rely on.
> Shouldn't we have a way to wait for Qemu to release the resources at
> vdpa driver shutdown time?
If dpdk-vdpa get killed and restart, qemu will shutdown device and
unmap the resources independently.
>
> Also as on patch 1, please add a Fixes tag it you want it to be
> backported.
Agree to backport, but not a fix, I'll add cc:stable@dpdk.org, the
patch will be noticed by maintainer, thanks for the suggestion!
>
> Regards,
> Maxime
>
next prev parent reply other threads:[~2021-10-13 10:14 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-23 8:17 [dpdk-dev] [PATCH 1/2] vdpa/mlx5: workaround FW first completion in start Xueming Li
2021-09-23 8:17 ` [dpdk-dev] [PATCH 2/2] vdpa/mlx5: retry VAR allocation during vDPA restart Xueming Li
2021-10-13 10:06 ` Maxime Coquelin
2021-10-13 10:14 ` Xueming(Steven) Li [this message]
2021-10-13 9:55 ` [dpdk-dev] [PATCH 1/2] vdpa/mlx5: workaround FW first completion in start Maxime Coquelin
2021-10-15 13:43 ` [dpdk-dev] [PATCH v1 " Xueming Li
2021-10-15 13:43 ` [dpdk-dev] [PATCH v1 2/2] vdpa/mlx5: retry VAR allocation during vDPA restart Xueming Li
2021-10-15 13:57 ` [dpdk-dev] [PATCH v1 1/2] vdpa/mlx5: workaround FW first completion in start Maxime Coquelin
2021-10-15 14:51 ` Xueming(Steven) Li
2021-10-15 15:05 ` [dpdk-dev] [PATCH v2 " Xueming Li
2021-10-15 15:05 ` [dpdk-dev] [PATCH v2 2/2] vdpa/mlx5: retry VAR allocation during vDPA restart Xueming Li
2021-10-21 9:40 ` Maxime Coquelin
2021-10-21 12:27 ` Maxime Coquelin
2021-10-21 9:40 ` [dpdk-dev] [PATCH v2 1/2] vdpa/mlx5: workaround FW first completion in start Maxime Coquelin
2021-10-21 12:27 ` Maxime Coquelin
2021-10-21 12:36 ` Xueming(Steven) Li
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=de4de5021946242f89b4d742d9b85574a63fc79b.camel@nvidia.com \
--to=xuemingl@nvidia.com \
--cc=dev@dpdk.org \
--cc=matan@nvidia.com \
--cc=maxime.coquelin@redhat.com \
--cc=viacheslavo@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).