From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 509ADA0C43; Wed, 12 May 2021 12:14:35 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 619E041108; Wed, 12 May 2021 12:14:27 +0200 (CEST) Received: from mail-lf1-f49.google.com (mail-lf1-f49.google.com [209.85.167.49]) by mails.dpdk.org (Postfix) with ESMTP id B366C410E3 for ; Wed, 12 May 2021 12:14:24 +0200 (CEST) Received: by mail-lf1-f49.google.com with SMTP id z13so32969150lft.1 for ; Wed, 12 May 2021 03:14:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TmWh8Wp2qjam8VbJM6dqam/gokaKg1o7pgexaceylcI=; b=YCB6/d5xWtJgobZz6qZRv9R7M33R3Co0qwf4Gly5vOQkBxFtk1Q3bTWQjl8mybw/MJ 5ZlhLxwdq2vNiLhNQx6I7IcvIGC78Wljvyo9X9AwEk6S2MWGTxG2we58QDk5wCAARp4K 3qqKb/+qHObtf653bwxZ5Ilxck9MEE+qXpqMwCpILwiLAA2PlU6fJ5eYJiHoGrXW1lkr SN3G0uYJcOcltsZNyeVgFGVjxgq4tNLsGaEouUu1/dZN8i4NI4rnk0OotTPXpaQnPFiK 0qj8pIAEDkDRX/lKoj2gVYJctDI9VPnvSZ6r0Leg/7oUq9Z+0HKQGNnpQ76Q10SkPszg 3jGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TmWh8Wp2qjam8VbJM6dqam/gokaKg1o7pgexaceylcI=; b=a2D+DLeklqXIi4+dAa1Fa9JneF1Dp54zdoe+zZSW4toGIP0zEdYz5hCcJlBn22sBgN gtQnx2jq5Q9FP2IhwKyN4VX6X4/R3vBghjttCewNq/vn7KJTRw868+z+8HL53TxmhAXb 2rQd7uKfY4MXb0k+lkOngaVni8GdWhQxTF/g1EAn0H3epE+y/KmrrZjdmjQr9aH6Z4Ru fpyUgF9XWgpB/y9E1F+qEGUrw/PRmnkXlkDQVQ6unCu1sRLU3JtoY8jotWRN5Zh9Ygf8 G2Sd630TQSKWJpiROWGTsaTB5c+6mRRH12278np6fpNCMF1V7X5WUganDkYBvmI5iYnx VKgg== X-Gm-Message-State: AOAM531RgntnOYaM+9r2d7HHDT4/GhGgRWCzJfdua1KKiTbRLxXl8IXw pWAQZFKHa405RyVeKge3gLA0HA== X-Google-Smtp-Source: ABdhPJyN6qYg4+b85g5IMZIXSJ8/peQeT62BXEv8M3ODzFA8wzJRsuAjNanZVSlwZSwHJCG7IeHgbg== X-Received: by 2002:ac2:57c1:: with SMTP id k1mr24978454lfo.453.1620814464382; Wed, 12 May 2021 03:14:24 -0700 (PDT) Received: from DESKTOP-U5LNN3J.localdomain (89-79-189-199.dynamic.chello.pl. [89.79.189.199]) by smtp.gmail.com with ESMTPSA id d26sm2892244lfa.118.2021.05.12.03.14.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 May 2021 03:14:23 -0700 (PDT) From: Michal Krawczyk To: ferruh.yigit@intel.com Cc: dev@dpdk.org, ndagan@amazon.com, gtzalik@amazon.com, igorch@amazon.com, upstream@semihalf.com, Stanislaw Kardach , Michal Krawczyk , Shay Agroskin Date: Wed, 12 May 2021 12:13:45 +0200 Message-Id: <20210512101346.24803-3-mk@semihalf.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210512101346.24803-1-mk@semihalf.com> References: <20210512101346.24803-1-mk@semihalf.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v5 2/3] net/ena: make ethdev references multiprocess safe X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Stanislaw Kardach rte_pci_device and rte_eth_dev are process-local structures. Therefore ena_adapter::pdev and ena_adapter::rte_dev cannot be used universally. Both ena_timer_wd_callback and ena_interrupt_handler_rte needs access to the rte_eth_dev, but as they are being setup and executed in the primary process, it is safe to pass there the same pointer, which is used for the device configuration. In all other cases, except the eth_ena_dev_init(), the rte_eth_dev_data is used instead. Signed-off-by: Stanislaw Kardach Signed-off-by: Michal Krawczyk Reviewed-by: Igor Chauskin Reviewed-by: Shay Agroskin --- v5: * Remove adapter::port_id field. * Update commit log. v4: * Remove access to the "rte_eth_devices" and instead pass rte_eth_dev as an argument to all the primary process callbacks. * Fix commit heading style. drivers/net/ena/ena_ethdev.c | 70 ++++++++++++++++++------------------ drivers/net/ena/ena_ethdev.h | 4 +-- 2 files changed, 36 insertions(+), 38 deletions(-) diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index fb2b19cfdb..e310e7ae8c 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -164,6 +164,7 @@ static const struct rte_pci_id pci_id_ena_map[] = { static struct ena_aenq_handlers aenq_handlers; static int ena_device_init(struct ena_com_dev *ena_dev, + struct rte_pci_device *pdev, struct ena_com_dev_get_features_ctx *get_feat_ctx, bool *wd_state); static int ena_dev_configure(struct rte_eth_dev *dev); @@ -457,11 +458,11 @@ static void ena_config_host_info(struct ena_com_dev *ena_dev) } /* This function calculates the number of xstats based on the current config */ -static unsigned int ena_xstats_calc_num(struct rte_eth_dev *dev) +static unsigned int ena_xstats_calc_num(struct rte_eth_dev_data *data) { return ENA_STATS_ARRAY_GLOBAL + ENA_STATS_ARRAY_ENI + - (dev->data->nb_tx_queues * ENA_STATS_ARRAY_TX) + - (dev->data->nb_rx_queues * ENA_STATS_ARRAY_RX); + (data->nb_tx_queues * ENA_STATS_ARRAY_TX) + + (data->nb_rx_queues * ENA_STATS_ARRAY_RX); } static void ena_config_debug_area(struct ena_adapter *adapter) @@ -469,7 +470,7 @@ static void ena_config_debug_area(struct ena_adapter *adapter) u32 debug_area_size; int rc, ss_count; - ss_count = ena_xstats_calc_num(adapter->rte_dev); + ss_count = ena_xstats_calc_num(adapter->edev_data); /* allocate 32 bytes for each string and 64bit for the value */ debug_area_size = ss_count * ETH_GSTRING_LEN + sizeof(u64) * ss_count; @@ -518,7 +519,7 @@ static int ena_close(struct rte_eth_dev *dev) rte_intr_disable(intr_handle); rte_intr_callback_unregister(intr_handle, ena_interrupt_handler_rte, - adapter); + dev); /* * MAC is not allocated dynamically. Setting NULL should prevent from @@ -593,7 +594,7 @@ static int ena_rss_reta_update(struct rte_eth_dev *dev, } PMD_DRV_LOG(DEBUG, "%s(): RSS configured %d entries for port %d\n", - __func__, reta_size, adapter->rte_dev->data->port_id); + __func__, reta_size, dev->data->port_id); return 0; } @@ -637,7 +638,7 @@ static int ena_rss_reta_query(struct rte_eth_dev *dev, static int ena_rss_init_default(struct ena_adapter *adapter) { struct ena_com_dev *ena_dev = &adapter->ena_dev; - uint16_t nb_rx_queues = adapter->rte_dev->data->nb_rx_queues; + uint16_t nb_rx_queues = adapter->edev_data->nb_rx_queues; int rc, i; u32 val; @@ -676,7 +677,7 @@ static int ena_rss_init_default(struct ena_adapter *adapter) goto err_fill_indir; } PMD_DRV_LOG(DEBUG, "RSS configured for port %d\n", - adapter->rte_dev->data->port_id); + adapter->edev_data->port_id); return 0; @@ -847,10 +848,10 @@ static uint32_t ena_get_mtu_conf(struct ena_adapter *adapter) { uint32_t max_frame_len = adapter->max_mtu; - if (adapter->rte_eth_dev_data->dev_conf.rxmode.offloads & + if (adapter->edev_data->dev_conf.rxmode.offloads & DEV_RX_OFFLOAD_JUMBO_FRAME) max_frame_len = - adapter->rte_eth_dev_data->dev_conf.rxmode.max_rx_pkt_len; + adapter->edev_data->dev_conf.rxmode.max_rx_pkt_len; return max_frame_len; } @@ -1070,8 +1071,8 @@ static int ena_start(struct rte_eth_dev *dev) if (rc) goto err_start_tx; - if (adapter->rte_dev->data->dev_conf.rxmode.mq_mode & - ETH_MQ_RX_RSS_FLAG && adapter->rte_dev->data->nb_rx_queues > 0) { + if (adapter->edev_data->dev_conf.rxmode.mq_mode & + ETH_MQ_RX_RSS_FLAG && adapter->edev_data->nb_rx_queues > 0) { rc = ena_rss_init_default(adapter); if (rc) goto err_rss_init; @@ -1084,7 +1085,7 @@ static int ena_start(struct rte_eth_dev *dev) ticks = rte_get_timer_hz(); rte_timer_reset(&adapter->timer_wd, ticks, PERIODICAL, rte_lcore_id(), - ena_timer_wd_callback, adapter); + ena_timer_wd_callback, dev); ++adapter->dev_stats.dev_start; adapter->state = ENA_ADAPTER_STATE_RUNNING; @@ -1505,6 +1506,7 @@ static int ena_populate_rx_queue(struct ena_ring *rxq, unsigned int count) } static int ena_device_init(struct ena_com_dev *ena_dev, + struct rte_pci_device *pdev, struct ena_com_dev_get_features_ctx *get_feat_ctx, bool *wd_state) { @@ -1522,9 +1524,7 @@ static int ena_device_init(struct ena_com_dev *ena_dev, /* The PCIe configuration space revision id indicate if mmio reg * read is disabled. */ - readless_supported = - !(((struct rte_pci_device *)ena_dev->dmadev)->id.class_id - & ENA_MMIO_DISABLE_REG_READ); + readless_supported = !(pdev->id.class_id & ENA_MMIO_DISABLE_REG_READ); ena_com_set_mmio_read_mode(ena_dev, readless_supported); /* reset device */ @@ -1595,12 +1595,13 @@ static int ena_device_init(struct ena_com_dev *ena_dev, static void ena_interrupt_handler_rte(void *cb_arg) { - struct ena_adapter *adapter = cb_arg; + struct rte_eth_dev *dev = cb_arg; + struct ena_adapter *adapter = dev->data->dev_private; struct ena_com_dev *ena_dev = &adapter->ena_dev; ena_com_admin_q_comp_intr_handler(ena_dev); if (likely(adapter->state != ENA_ADAPTER_STATE_CLOSED)) - ena_com_aenq_intr_handler(ena_dev, adapter); + ena_com_aenq_intr_handler(ena_dev, dev); } static void check_for_missing_keep_alive(struct ena_adapter *adapter) @@ -1633,8 +1634,8 @@ static void check_for_admin_com_state(struct ena_adapter *adapter) static void ena_timer_wd_callback(__rte_unused struct rte_timer *timer, void *arg) { - struct ena_adapter *adapter = arg; - struct rte_eth_dev *dev = adapter->rte_dev; + struct rte_eth_dev *dev = arg; + struct ena_adapter *adapter = dev->data->dev_private; check_for_missing_keep_alive(adapter); check_for_admin_com_state(adapter); @@ -1775,11 +1776,9 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev) memset(adapter, 0, sizeof(struct ena_adapter)); ena_dev = &adapter->ena_dev; - adapter->rte_eth_dev_data = eth_dev->data; - adapter->rte_dev = eth_dev; + adapter->edev_data = eth_dev->data; pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); - adapter->pdev = pci_dev; PMD_INIT_LOG(INFO, "Initializing %x:%x:%x.%d", pci_dev->addr.domain, @@ -1799,7 +1798,8 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev) } ena_dev->reg_bar = adapter->regs; - ena_dev->dmadev = adapter->pdev; + /* This is a dummy pointer for ena_com functions. */ + ena_dev->dmadev = adapter; adapter->id_number = adapters_found; @@ -1813,7 +1813,7 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev) } /* device specific initialization routine */ - rc = ena_device_init(ena_dev, &get_feat_ctx, &wd_state); + rc = ena_device_init(ena_dev, pci_dev, &get_feat_ctx, &wd_state); if (rc) { PMD_INIT_LOG(CRIT, "Failed to init ENA device"); goto err; @@ -1896,7 +1896,7 @@ static int eth_ena_dev_init(struct rte_eth_dev *eth_dev) rte_intr_callback_register(intr_handle, ena_interrupt_handler_rte, - adapter); + eth_dev); rte_intr_enable(intr_handle); ena_com_set_admin_polling_mode(ena_dev, false); ena_com_admin_aenq_enable(ena_dev); @@ -2679,7 +2679,7 @@ static int ena_xstats_get_names(struct rte_eth_dev *dev, struct rte_eth_xstat_name *xstats_names, unsigned int n) { - unsigned int xstats_count = ena_xstats_calc_num(dev); + unsigned int xstats_count = ena_xstats_calc_num(dev->data); unsigned int stat, i, count = 0; if (n < xstats_count || !xstats_names) @@ -2728,7 +2728,7 @@ static int ena_xstats_get(struct rte_eth_dev *dev, unsigned int n) { struct ena_adapter *adapter = dev->data->dev_private; - unsigned int xstats_count = ena_xstats_calc_num(dev); + unsigned int xstats_count = ena_xstats_calc_num(dev->data); unsigned int stat, i, count = 0; int stat_offset; void *stats_begin; @@ -2953,14 +2953,12 @@ RTE_LOG_REGISTER_SUFFIX(ena_logtype_com, com, NOTICE); static void ena_update_on_link_change(void *adapter_data, struct ena_admin_aenq_entry *aenq_e) { - struct rte_eth_dev *eth_dev; - struct ena_adapter *adapter; + struct rte_eth_dev *eth_dev = adapter_data; + struct ena_adapter *adapter = eth_dev->data->dev_private; struct ena_admin_aenq_link_change_desc *aenq_link_desc; uint32_t status; - adapter = adapter_data; aenq_link_desc = (struct ena_admin_aenq_link_change_desc *)aenq_e; - eth_dev = adapter->rte_dev; status = get_ena_admin_aenq_link_change_desc_link_status(aenq_link_desc); adapter->link_status = status; @@ -2969,10 +2967,11 @@ static void ena_update_on_link_change(void *adapter_data, rte_eth_dev_callback_process(eth_dev, RTE_ETH_EVENT_INTR_LSC, NULL); } -static void ena_notification(void *data, +static void ena_notification(void *adapter_data, struct ena_admin_aenq_entry *aenq_e) { - struct ena_adapter *adapter = data; + struct rte_eth_dev *eth_dev = adapter_data; + struct ena_adapter *adapter = eth_dev->data->dev_private; struct ena_admin_ena_hw_hints *hints; if (aenq_e->aenq_common_desc.group != ENA_ADMIN_NOTIFICATION) @@ -2995,7 +2994,8 @@ static void ena_notification(void *data, static void ena_keep_alive(void *adapter_data, __rte_unused struct ena_admin_aenq_entry *aenq_e) { - struct ena_adapter *adapter = adapter_data; + struct rte_eth_dev *eth_dev = adapter_data; + struct ena_adapter *adapter = eth_dev->data->dev_private; struct ena_admin_aenq_keep_alive_desc *desc; uint64_t rx_drops; uint64_t tx_drops; diff --git a/drivers/net/ena/ena_ethdev.h b/drivers/net/ena/ena_ethdev.h index ae235897ee..78718b759b 100644 --- a/drivers/net/ena/ena_ethdev.h +++ b/drivers/net/ena/ena_ethdev.h @@ -210,9 +210,7 @@ struct ena_offloads { /* board specific private data structure */ struct ena_adapter { /* OS defined structs */ - struct rte_pci_device *pdev; - struct rte_eth_dev_data *rte_eth_dev_data; - struct rte_eth_dev *rte_dev; + struct rte_eth_dev_data *edev_data; struct ena_com_dev ena_dev __rte_cache_aligned; -- 2.25.1