RE: [PATCH 20.11] vdpa/mlx5: workaround var offset within page

["Xueming(Steven) Li" <xuemingl@nvidia.com>]

Thanks, applied!

> -----Original Message-----
> From: Yajun Wu <yajunw@nvidia.com>
> Sent: Thursday, July 21, 2022 1:43 PM
> To: stable@dpdk.org
> Cc: Matan Azrad <matan@nvidia.com>; Maxime Coquelin <maxime.coquelin@redhat.com>
> Subject: [PATCH 20.11] vdpa/mlx5: workaround var offset within page
> 
> [ upstream commit 95af59b7ad9f6a465de2ead9ef709429678e750d ]
> 
> vDPA driver first uses kernel driver to allocate doorbell(VAR) area for each device. Then uses var->mmap_off and var->length to mmap
> uverbs device file as doorbell userspace virtual address.
> 
> Current kernel driver provides var->mmap_off equal to page start of VAR.
> It's fine with x86 4K page server, because VAR physical address is only 4K aligned thus locate in 4K page start.
> 
> But with aarch64 64K page server, the actual VAR physical address has offset within page(not locate in 64K page start). So vDPA driver need
> add this within page offset(caps.doorbell_bar_offset) to get right VAR virtual address.
> 
> Fixes: 62c813706e4 ("vdpa/mlx5: map doorbell")
> 
> Signed-off-by: Yajun Wu <yajunw@nvidia.com>
> Acked-by: Matan Azrad <matan@nvidia.com>
> Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
>  drivers/vdpa/mlx5/mlx5_vdpa_virtq.c | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c b/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c
> index 0ef7ed0e4a..952e641425 100644
> --- a/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c
> +++ b/drivers/vdpa/mlx5/mlx5_vdpa_virtq.c
> @@ -9,6 +9,7 @@
>  #include <rte_malloc.h>
>  #include <rte_errno.h>
>  #include <rte_io.h>
> +#include <rte_eal_paging.h>
> 
>  #include <mlx5_common.h>
> 
> @@ -122,7 +123,9 @@ mlx5_vdpa_virtqs_release(struct mlx5_vdpa_priv *priv)
>  		priv->td = NULL;
>  	}
>  	if (priv->virtq_db_addr) {
> -		claim_zero(munmap(priv->virtq_db_addr, priv->var->length));
> +		/* Mask out the within page offset for munmap. */
> +		claim_zero(munmap((void *)((uintptr_t)priv->virtq_db_addr &
> +			~(rte_mem_page_size() - 1)), priv->var->length));
>  		priv->virtq_db_addr = NULL;
>  	}
>  	priv->features = 0;
> @@ -485,6 +488,10 @@ mlx5_vdpa_virtqs_prepare(struct mlx5_vdpa_priv *priv)
>  		priv->virtq_db_addr = NULL;
>  		goto error;
>  	} else {
> +		/* Add within page offset for 64K page system. */
> +		priv->virtq_db_addr = (char *)priv->virtq_db_addr +
> +			((rte_mem_page_size() - 1) &
> +			priv->caps.doorbell_bar_offset);
>  		DRV_LOG(DEBUG, "VAR address of doorbell mapping is %p.",
>  			priv->virtq_db_addr);
>  	}
> --
> 2.27.0