DPDK patches and discussions
 help / color / mirror / Atom feed
From: Matan Azrad <matan@nvidia.com>
To: Gregory Etelson <getelson@nvidia.com>, "dev@dpdk.org" <dev@dpdk.org>
Cc: Maayan Kashani <mkashani@nvidia.com>,
	Raslan Darawsheh <rasland@nvidia.com>,
	Michael Baum <michaelba@nvidia.com>,
	Dariusz Sosnowski <dsosnowski@nvidia.com>,
	Slava Ovsiienko <viacheslavo@nvidia.com>,
	Bing Zhao <bingz@nvidia.com>, Ori Kam <orika@nvidia.com>,
	Suanming Mou <suanmingm@nvidia.com>
Subject: RE: [PATCH 1/2] common/mlx5: add device duplication function
Date: Mon, 17 Mar 2025 10:47:54 +0000	[thread overview]
Message-ID: <SA3PR12MB8762821F93E85C58DAE30DF5DFDF2@SA3PR12MB8762.namprd12.prod.outlook.com> (raw)
In-Reply-To: <20250313083351.25559-1-getelson@nvidia.com>



> From: Michael Baum <michaelba@nvidia.com>
> 
> Add function for creating additional CTX for same device base on existing CTX.
> 
> Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>


> ---
>  drivers/common/mlx5/linux/mlx5_common_os.c   | 52
> ++++++++++++++++++++
>  drivers/common/mlx5/mlx5_common.h            |  4 ++
>  drivers/common/mlx5/version.map              |  1 +
>  drivers/common/mlx5/windows/mlx5_common_os.c | 31 ++++++++++++
>  4 files changed, 88 insertions(+)
> 
> diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c
> b/drivers/common/mlx5/linux/mlx5_common_os.c
> index 41345e1597..2f5032f0da 100644
> --- a/drivers/common/mlx5/linux/mlx5_common_os.c
> +++ b/drivers/common/mlx5/linux/mlx5_common_os.c
> @@ -873,6 +873,58 @@ mlx5_os_open_device(struct mlx5_common_device
> *cdev, uint32_t classes)
>  	return 0;
>  }
> 
> +/**
> + * API function to obtain a new InfiniBand (IB) context for a given common
> device.
> + *
> + * This function provides a port-agnostic IB context for a physical
> +device, enabling the
> + * device to create and manage resources that can be initialized when a
> +port starts and
> + * released when another port stops.
> + *
> + * For Linux, it imports new context from the existing context.
> + *
> + * @param cdev
> + *   Pointer to the mlx5 device structure.
> + *
> + * @return
> + *   Pointer to an `ibv_context` on success, or NULL on failure, with `rte_errno`
> set.
> + */
> +void *
> +mlx5_os_get_physical_device_ctx(struct mlx5_common_device *cdev) {
> +	struct ibv_context *ctx = NULL;
> +	int cmd_fd = ((struct ibv_context *)cdev->ctx)->cmd_fd;
> +	int new_cmd_fd;
> +
> +	/*
> +	 * Duplicate the command FD to pass it as input to the import device
> function.
> +	 * If the import function succeeds, the new device context takes
> ownership of
> +	 * this FD, which will be freed when the new device is closed.
> +	 * If the import function fails, we are responsible for closing this FD.
> +	 */
> +	new_cmd_fd = dup(cmd_fd);
> +	if (new_cmd_fd < 0) {
> +		DRV_LOG(ERR,
> +			"Failed to duplicate FD %d for IB device \"%s\": %s",
> +			cmd_fd, mlx5_os_get_ctx_device_name(cdev->ctx),
> +			rte_strerror(errno));
> +		rte_errno = errno;
> +		return NULL;
> +	}
> +	/* Attempt to import the duplicated FD to create a new device context.
> */
> +	ctx = mlx5_glue->import_device(new_cmd_fd);
> +	if (!ctx) {
> +		DRV_LOG(ERR, "Failed to import IB device \"%s\": %s",
> +			mlx5_os_get_ctx_device_name(cdev->ctx),
> +			rte_strerror(errno));
> +		close(new_cmd_fd);
> +		rte_errno = errno;
> +		return NULL;
> +	}
> +	DRV_LOG(INFO, "IB device \"%s\" successfully imported, old_fd=%d,
> new_fd=%d",
> +		mlx5_os_get_ctx_device_name(cdev->ctx), cmd_fd,
> new_cmd_fd);
> +	return (void *)ctx;
> +}
> +
>  int
>  mlx5_get_device_guid(const struct rte_pci_addr *dev, uint8_t *guid, size_t
> len)  { diff --git a/drivers/common/mlx5/mlx5_common.h
> b/drivers/common/mlx5/mlx5_common.h
> index e7bd4c6ec4..bea1382911 100644
> --- a/drivers/common/mlx5/mlx5_common.h
> +++ b/drivers/common/mlx5/mlx5_common.h
> @@ -651,6 +651,10 @@ mlx5_devx_uar_release(struct mlx5_uar *uar);
> 
>  /* mlx5_common_os.c */
> 
> +__rte_internal
> +void *
> +mlx5_os_get_physical_device_ctx(struct mlx5_common_device *cdev);
> +
>  int mlx5_os_open_device(struct mlx5_common_device *cdev, uint32_t
> classes);  int mlx5_os_pd_prepare(struct mlx5_common_device *cdev);  int
> mlx5_os_pd_release(struct mlx5_common_device *cdev); diff --git
> a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map
> index 8301485839..819e6b96cb 100644
> --- a/drivers/common/mlx5/version.map
> +++ b/drivers/common/mlx5/version.map
> @@ -150,6 +150,7 @@ INTERNAL {
>  	mlx5_nl_rdma_monitor_info_get; # WINDOWS_NO_EXPORT
>  	mlx5_nl_rdma_monitor_cap_get; # WINDOWS_NO_EXPORT
> 
> +	mlx5_os_get_physical_device_ctx;
>  	mlx5_os_umem_dereg;
>  	mlx5_os_umem_reg;
> 
> diff --git a/drivers/common/mlx5/windows/mlx5_common_os.c
> b/drivers/common/mlx5/windows/mlx5_common_os.c
> index 2b6058eb3e..68f1981193 100644
> --- a/drivers/common/mlx5/windows/mlx5_common_os.c
> +++ b/drivers/common/mlx5/windows/mlx5_common_os.c
> @@ -266,6 +266,37 @@ mlx5_os_open_device(struct mlx5_common_device
> *cdev, uint32_t classes)
>  	return -rte_errno;
>  }
> 
> +/**
> + * API function to obtain a new MLX5 context for a given common device.
> + *
> + * This function provides a port-agnostic context for a physical
> +device, enabling the
> + * device to create and manage resources that can be initialized when a
> +port starts and
> + * released when another port stops.
> + *
> + * For Windows, it creates a new context for the device regardless to existing
> context.
> + *
> + * @param cdev
> + *   Pointer to the mlx5 device structure.
> + *
> + * @return
> + *   Pointer to an `ibv_context` on success, or NULL on failure, with `rte_errno`
> set.
> + */
> +void *
> +mlx5_os_get_physical_device_ctx(struct mlx5_common_device *cdev) {
> +	struct mlx5_common_device temp = {
> +		.dev = cdev->dev,
> +	};
> +
> +	if (mlx5_os_open_device(&temp, MLX5_CLASS_ETH) < 0) {
> +		DRV_LOG(ERR, "Failed to duplicate DevX device \"%s\": %s",
> +			mlx5_os_get_ctx_device_name(cdev->ctx),
> +			rte_strerror(rte_errno));
> +		return NULL;
> +	}
> +	return (void *)temp.ctx;
> +}
> +
>  /**
>   * Register umem.
>   *
> --
> 2.45.2


  parent reply	other threads:[~2025-03-17 10:47 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-03-13  8:33 Gregory Etelson
2025-03-13  8:33 ` [PATCH 2/2] net/mlx5: fix GENEVE parser cleanup Gregory Etelson
2025-03-17 10:48   ` Matan Azrad
2025-03-17 10:47 ` Matan Azrad [this message]
2025-03-18 10:38 ` [PATCH 1/2] common/mlx5: add device duplication function Raslan Darawsheh

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=SA3PR12MB8762821F93E85C58DAE30DF5DFDF2@SA3PR12MB8762.namprd12.prod.outlook.com \
    --to=matan@nvidia.com \
    --cc=bingz@nvidia.com \
    --cc=dev@dpdk.org \
    --cc=dsosnowski@nvidia.com \
    --cc=getelson@nvidia.com \
    --cc=michaelba@nvidia.com \
    --cc=mkashani@nvidia.com \
    --cc=orika@nvidia.com \
    --cc=rasland@nvidia.com \
    --cc=suanmingm@nvidia.com \
    --cc=viacheslavo@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).