From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6373DA034F; Fri, 8 Oct 2021 12:57:17 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4D1B64117F; Fri, 8 Oct 2021 12:56:13 +0200 (CEST) Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) by mails.dpdk.org (Postfix) with ESMTP id 0F05D411E0 for ; Thu, 7 Oct 2021 20:44:52 +0200 (CEST) Received: by mail-pj1-f48.google.com with SMTP id ls14-20020a17090b350e00b001a00e2251c8so5832479pjb.4 for ; Thu, 07 Oct 2021 11:44:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oneconvergence.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TqaLTr2FEirTzgXk1Ixyd2oWhuZS7xQXzt0WLyuC/pg=; b=gAzguV6GUXjfU7wGts7onmK5OizR08mMSvXp1FFrt3JGw1tMBWyuJ+wq8y01S6Izmp Kmco3n8C3sVkmjqo193t073uyQwkTY4R6kkn1W4NlLkjVWjMi/FmnVzCNlsPWWhObZSI 7j8FQkzKdnE48eyJuPUsvJlMOLF9eQTixxs3g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TqaLTr2FEirTzgXk1Ixyd2oWhuZS7xQXzt0WLyuC/pg=; b=LciER8eSseG3hk6C3NzF+zSwpU7IXLdjcF9V1FxwfaP7b9rC3YlIp7b3O/sMBrjhMT Q2BgjKz+4D46bvyJfrfbJ/H7xZV9jSMlUO3vJs4Bxy4QfBcfZE7yNODf5rXw5gHwzLKk lEQIGNQN739hm5gEX23PN5al3MQkgniR30o+9gxzKOB1CiGnMzfl/dAoEcc9MalnJwWa KjYeUHQqiY7lNiYSA7mzSeAfG2is5NXUlKpawfUX51JN5fNcGx72I8tALDILRbFw19p0 PZDmn8kLWtsvnp66Ev8FbAm9bMPAn+tT7+ChOZrrXdC3QUVbXQqhi1huqP29hfH6AvfJ 5ThQ== X-Gm-Message-State: AOAM533hKnayqjNpvtKTWP4xh1YG3HJ7sXT6pffRQhCxVbntjjAXT0h9 DvZOfJ5unaTj5br32WoAuJC8ng== X-Google-Smtp-Source: ABdhPJzkPrZbG54oTYqS8J2k+aldzdV0dJ8UDSH9fdTmqXGDmb51Xoqco+WZnkXv7jyF6bIWD0wnrQ== X-Received: by 2002:a17:90a:e38f:: with SMTP id b15mr6968818pjz.76.1633632291214; Thu, 07 Oct 2021 11:44:51 -0700 (PDT) Received: from srikanth-ThinkPad-T450.domain.name ([136.185.113.102]) by smtp.gmail.com with ESMTPSA id c11sm3311586pji.38.2021.10.07.11.44.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Oct 2021 11:44:50 -0700 (PDT) From: Srikanth Kaka To: Matan Azrad , Viacheslav Ovsiienko Cc: dev@dpdk.org, Vag Singh , Anand Thulasiram , Srikanth Kaka Date: Fri, 8 Oct 2021 00:13:23 +0530 Message-Id: <20211007184350.73858-15-srikanth.k@oneconvergence.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211007184350.73858-1-srikanth.k@oneconvergence.com> References: <20211007184350.73858-1-srikanth.k@oneconvergence.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Mailman-Approved-At: Fri, 08 Oct 2021 12:55:54 +0200 Subject: [dpdk-dev] [PATCH v2 14/41] net/mlx5: add open IB device routines X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The mlx5_os_open_device(), mlx5_config_doorbell_mapping_env(), mlx5_restore_doorbell_mapping_env(), mlx5_alloc_verbs_buf() and mlx5_free_verbs_buf() are equivalent to Linux APIs of the same name Signed-off-by: Srikanth Kaka Signed-off-by: Vag Singh Signed-off-by: Anand Thulasiram --- drivers/net/mlx5/freebsd/mlx5_os.c | 155 +++++++++++++++++++++++++++++ 1 file changed, 155 insertions(+) diff --git a/drivers/net/mlx5/freebsd/mlx5_os.c b/drivers/net/mlx5/freebsd/mlx5_os.c index 1d8e627988..53f74b9d85 100644 --- a/drivers/net/mlx5/freebsd/mlx5_os.c +++ b/drivers/net/mlx5/freebsd/mlx5_os.c @@ -5,16 +5,28 @@ */ #include +#include +#include #include #include #include +#include +#include +#include + #include +#include #include +#include #include "mlx5_defs.h" #include "mlx5.h" +#include "mlx5_common_os.h" +#include "mlx5_rxtx.h" #include "mlx5_autoconf.h" +#include "mlx5_mr.h" +#include "mlx5_verbs.h" /** * Get mlx5 device attributes. The glue function query_device_ex() is called @@ -86,6 +98,149 @@ mlx5_os_get_dev_attr(void *ctx, struct mlx5_dev_attr *device_attr) return err; } +/** + * Verbs callback to allocate a memory. This function should allocate the space + * according to the size provided residing inside a huge page. + * Please note that all allocation must respect the alignment from libmlx5 + * (i.e. currently rte_mem_page_size()). + * + * @param[in] size + * The size in bytes of the memory to allocate. + * @param[in] data + * A pointer to the callback data. + * + * @return + * Allocated buffer, NULL otherwise and rte_errno is set. + */ +static void * +mlx5_alloc_verbs_buf(size_t size, void *data) +{ + struct mlx5_dev_ctx_shared *sh = data; + void *ret; + size_t alignment = rte_mem_page_size(); + if (alignment == (size_t)-1) { + DRV_LOG(ERR, "Failed to get mem page size"); + rte_errno = ENOMEM; + return NULL; + } + + MLX5_ASSERT(data != NULL); + ret = mlx5_malloc(0, size, alignment, sh->numa_node); + if (!ret && size) + rte_errno = ENOMEM; + return ret; +} + +/** + * Verbs callback to free a memory. + * + * @param[in] ptr + * A pointer to the memory to free. + * @param[in] data + * A pointer to the callback data. + */ +static void +mlx5_free_verbs_buf(void *ptr, void *data __rte_unused) +{ + MLX5_ASSERT(data != NULL); + mlx5_free(ptr); +} + +static int +mlx5_config_doorbell_mapping_env(const struct mlx5_dev_config *config) +{ + char *env; + int value; + + MLX5_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY); + /* Get environment variable to store. */ + env = getenv(MLX5_SHUT_UP_BF); + value = env ? !!strcmp(env, "0") : MLX5_ARG_UNSET; + if (config->dbnc == MLX5_ARG_UNSET) + setenv(MLX5_SHUT_UP_BF, MLX5_SHUT_UP_BF_DEFAULT, 1); + else + setenv(MLX5_SHUT_UP_BF, + config->dbnc == MLX5_TXDB_NCACHED ? "1" : "0", 1); + return value; +} + +static void +mlx5_restore_doorbell_mapping_env(int value) +{ + MLX5_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY); + /* Restore the original environment variable state. */ + if (value == MLX5_ARG_UNSET) + unsetenv(MLX5_SHUT_UP_BF); + else + setenv(MLX5_SHUT_UP_BF, value ? "1" : "0", 1); +} + +/** + * Function API to open IB device. + * + * This function calls the Linux glue APIs to open a device. + * + * @param[in] spawn + * Pointer to the IB device attributes (name, port, etc). + * @param[out] config + * Pointer to device configuration structure. + * @param[out] sh + * Pointer to shared context structure. + * + * @return + * 0 on success, a positive error value otherwise. + */ +int +mlx5_os_open_device(const struct mlx5_dev_spawn_data *spawn, + const struct mlx5_dev_config *config, + struct mlx5_dev_ctx_shared *sh) +{ + int dbmap_env; + int err = 0; + + pthread_mutex_init(&sh->txpp.mutex, NULL); + /* + * Configure environment variable "MLX5_BF_SHUT_UP" + * before the device creation. The rdma_core library + * checks the variable at device creation and + * stores the result internally. + */ + dbmap_env = mlx5_config_doorbell_mapping_env(config); + /* Try to open IB device with DV first, then usual Verbs. */ + errno = 0; + sh->ctx = mlx5_glue->dv_open_device(spawn->phys_dev); + if (sh->ctx) { + sh->devx = 1; + DRV_LOG(DEBUG, "DevX is supported"); + /* The device is created, no need for environment. */ + mlx5_restore_doorbell_mapping_env(dbmap_env); + } else { + /* The environment variable is still configured. */ + sh->ctx = mlx5_glue->open_device(spawn->phys_dev); + err = errno ? errno : ENODEV; + /* + * The environment variable is not needed anymore, + * all device creation attempts are completed. + */ + mlx5_restore_doorbell_mapping_env(dbmap_env); + if (!sh->ctx) + return err; + DRV_LOG(DEBUG, "DevX is NOT supported"); + err = 0; + } + if (!err && sh->ctx) { + /* Hint libmlx5 to use PMD allocator for data plane resources */ + mlx5_glue->dv_set_context_attr(sh->ctx, + MLX5DV_CTX_ATTR_BUF_ALLOCATORS, + (void *)((uintptr_t)&(struct mlx5dv_ctx_allocators){ + .alloc = &mlx5_alloc_verbs_buf, + .free = &mlx5_free_verbs_buf, + .data = sh, + })); + } + return err; +} + /** * Set the reg_mr and dereg_mr call backs * -- 2.30.2