From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4066042C46; Wed, 7 Jun 2023 04:46:10 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2A37D410F6; Wed, 7 Jun 2023 04:46:10 +0200 (CEST) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by mails.dpdk.org (Postfix) with ESMTP id B72DA40A84; Wed, 7 Jun 2023 04:46:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686105969; x=1717641969; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Wb7L/x3nyRzcJa0I9OMSQcuxTHfz7IS4ESyyTB/8uYc=; b=DSRj460yiwXrraqFni0s1I5um9U41wBfz7JRJh4Eo9pspYK268he5lbS lQM/YA6+xACAoNURqaW0C6b4oIbV4QIWOnC/zvBZg0cew7XAeF0uHBsOP WnFqDFAJ7zJ8XcuS71cDfKRH5gUQfpZZuki6aFsm6YIymwIUAZGsLrD6/ bUN8WPPVExYFxELXhTRJEN1lbwd7MLXdzdQ6Tu3zw6P6fMZf0Dw8qP/sR f+UiPeuobvFHVe0AfxUZs0AgwpF7QjuzahutUwwxAVrmCvNZt7glnxYTD D74W+nNStUJXmI/dJQuL8kd4vE4hJzegoePAYqV8CjUuKnBo8mrxXmj3t g==; X-IronPort-AV: E=McAfee;i="6600,9927,10733"; a="337220479" X-IronPort-AV: E=Sophos;i="6.00,222,1681196400"; d="scan'208";a="337220479" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Jun 2023 19:46:07 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10733"; a="712420205" X-IronPort-AV: E=Sophos;i="6.00,222,1681196400"; d="scan'208";a="712420205" Received: from shwdenpg561.ccr.corp.intel.com (HELO dpdk..) ([10.239.252.3]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Jun 2023 19:46:04 -0700 From: Kaiwen Deng To: dev@dpdk.org Cc: stable@dpdk.org, qiming.yang@intel.com, yidingx.zhou@intel.com, Kaiwen Deng , Chas Williams , "Min Hu (Connor)" , Jingjing Wu , Beilei Xing , Qi Zhang , Mike Pattrick , Declan Doherty , Daniel Mrzyglod , Dapeng Yu Subject: [PATCH v4] net/iavf: fix iavf query stats in intr thread Date: Wed, 7 Jun 2023 10:03:50 +0800 Message-Id: <20230607020350.409080-1-kaiwenx.deng@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230322072615.179196-1-kaiwenx.deng@intel.com> References: <20230322072615.179196-1-kaiwenx.deng@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When iavf send query-stats command in eal-intr-thread through virtual channel, there will be no response received from iavf_dev_virtchnl_handler for this command during block and wait. Because iavf_dev_virtchnl_handler is also registered in eal-intr-thread. When vf device is bonded as BONDING_MODE_TLB mode, the slave device update callback will registered in alarm and called by eal-intr-thread, it would also raise the above issue. This commit add to poll the response for VIRTCHNL_OP_GET_STATS when it is called by eal-intr-thread to fix this issue. Fixes: 91bf37d250aa ("net/iavf: add lock for VF commands") Fixes: 22b123a36d07 ("net/avf: initialize PMD") Fixes: 7c76a747e68c ("bond: add mode 5") Fixes: 435d523112cc ("net/iavf: fix multi-process shared data") Fixes: cb5c1b91f76f ("net/iavf: add thread for event callbacks") Cc: stable@dpdk.org Signed-off-by: Kaiwen Deng --- Changes since v3: - Rebase to latest dpdk-next-net-intel. Changes since v2: - Add to poll the response for VIRTCHNL_OP_GET_STATS. Changes since v1: - Add lock to avoid race condition. --- --- drivers/net/bonding/rte_eth_bond_pmd.c | 7 ++- drivers/net/iavf/iavf_ethdev.c | 5 +- drivers/net/iavf/iavf_vchnl.c | 71 ++++++++++++++++++-------- 3 files changed, 58 insertions(+), 25 deletions(-) diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index f0c4f7d26b..edce621496 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -894,6 +894,7 @@ bond_ethdev_update_tlb_slave_cb(void *arg) uint8_t update_stats = 0; uint16_t slave_id; uint16_t i; + int ret; internals->slave_update_idx++; @@ -903,7 +904,10 @@ bond_ethdev_update_tlb_slave_cb(void *arg) for (i = 0; i < internals->active_slave_count; i++) { slave_id = internals->active_slaves[i]; - rte_eth_stats_get(slave_id, &slave_stats); + ret = rte_eth_stats_get(slave_id, &slave_stats); + if (ret) + goto OUT; + tx_bytes = slave_stats.obytes - tlb_last_obytets[slave_id]; bandwidth_left(slave_id, tx_bytes, internals->slave_update_idx, &bwg_array[i]); @@ -922,6 +926,7 @@ bond_ethdev_update_tlb_slave_cb(void *arg) for (i = 0; i < slave_count; i++) internals->tlb_slaves_order[i] = bwg_array[i].slave; +OUT: rte_eal_alarm_set(REORDER_PERIOD_MS * 1000, bond_ethdev_update_tlb_slave_cb, (struct bond_dev_private *)internals); } diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c index 96004220a1..b72dbc8ceb 100644 --- a/drivers/net/iavf/iavf_ethdev.c +++ b/drivers/net/iavf/iavf_ethdev.c @@ -2647,6 +2647,9 @@ iavf_dev_init(struct rte_eth_dev *eth_dev) adapter->dev_data = eth_dev->data; adapter->stopped = 1; + if (iavf_dev_event_handler_init()) + goto init_vf_err; + if (iavf_init_vf(eth_dev) != 0) { PMD_INIT_LOG(ERR, "Init vf failed"); return -1; @@ -2674,8 +2677,6 @@ iavf_dev_init(struct rte_eth_dev *eth_dev) rte_ether_addr_copy((struct rte_ether_addr *)hw->mac.addr, ð_dev->data->mac_addrs[0]); - if (iavf_dev_event_handler_init()) - goto init_vf_err; if (vf->vf_res->vf_cap_flags & VIRTCHNL_VF_OFFLOAD_WB_ON_ITR) { /* register callback func to eal lib */ diff --git a/drivers/net/iavf/iavf_vchnl.c b/drivers/net/iavf/iavf_vchnl.c index 8cc5377bcf..cb353db008 100644 --- a/drivers/net/iavf/iavf_vchnl.c +++ b/drivers/net/iavf/iavf_vchnl.c @@ -256,6 +256,7 @@ iavf_read_msg_from_pf(struct iavf_adapter *adapter, uint16_t buf_len, vf->link_speed = iavf_convert_link_speed(speed); } iavf_dev_link_update(vf->eth_dev, 0); + iavf_dev_event_post(vf->eth_dev, RTE_ETH_EVENT_INTR_LSC, NULL, 0); if (vf->link_up && !vf->vf_reset) { iavf_dev_watchdog_disable(adapter); } else { @@ -374,28 +375,48 @@ iavf_execute_vf_cmd(struct iavf_adapter *adapter, struct iavf_cmd_info *args, _clear_cmd(vf); break; default: - /* For other virtchnl ops in running time, - * wait for the cmd done flag. - */ - do { - if (vf->pend_cmd == VIRTCHNL_OP_UNKNOWN) - break; - iavf_msec_delay(ASQ_DELAY_MS); - /* If don't read msg or read sys event, continue */ - } while (i++ < MAX_TRY_TIMES); - - if (i >= MAX_TRY_TIMES) { - PMD_DRV_LOG(ERR, "No response for cmd %d", args->ops); + if (rte_thread_is_intr()) { + /* For virtchnl ops were executed in eal_intr_thread, + * need to poll the response. + */ + do { + result = iavf_read_msg_from_pf(adapter, args->out_size, + args->out_buffer); + if (result == IAVF_MSG_CMD) + break; + iavf_msec_delay(ASQ_DELAY_MS); + } while (i++ < MAX_TRY_TIMES); + if (i >= MAX_TRY_TIMES || + vf->cmd_retval != VIRTCHNL_STATUS_SUCCESS) { + err = -1; + PMD_DRV_LOG(ERR, "No response or return failure (%d)" + " for cmd %d", vf->cmd_retval, args->ops); + } _clear_cmd(vf); - err = -EIO; - } else if (vf->cmd_retval == - VIRTCHNL_STATUS_ERR_NOT_SUPPORTED) { - PMD_DRV_LOG(ERR, "Cmd %d not supported", args->ops); - err = -ENOTSUP; - } else if (vf->cmd_retval != VIRTCHNL_STATUS_SUCCESS) { - PMD_DRV_LOG(ERR, "Return failure %d for cmd %d", - vf->cmd_retval, args->ops); - err = -EINVAL; + } else { + /* For other virtchnl ops in running time, + * wait for the cmd done flag. + */ + do { + if (vf->pend_cmd == VIRTCHNL_OP_UNKNOWN) + break; + iavf_msec_delay(ASQ_DELAY_MS); + /* If don't read msg or read sys event, continue */ + } while (i++ < MAX_TRY_TIMES); + + if (i >= MAX_TRY_TIMES) { + PMD_DRV_LOG(ERR, "No response for cmd %d", args->ops); + _clear_cmd(vf); + err = -EIO; + } else if (vf->cmd_retval == + VIRTCHNL_STATUS_ERR_NOT_SUPPORTED) { + PMD_DRV_LOG(ERR, "Cmd %d not supported", args->ops); + err = -ENOTSUP; + } else if (vf->cmd_retval != VIRTCHNL_STATUS_SUCCESS) { + PMD_DRV_LOG(ERR, "Return failure %d for cmd %d", + vf->cmd_retval, args->ops); + err = -EINVAL; + } } break; } @@ -409,8 +430,14 @@ iavf_execute_vf_cmd_safe(struct iavf_adapter *adapter, { struct iavf_info *vf = IAVF_DEV_PRIVATE_TO_VF(adapter); int ret; + int is_intr_thread = rte_thread_is_intr(); - rte_spinlock_lock(&vf->aq_lock); + if (is_intr_thread) { + if (!rte_spinlock_trylock(&vf->aq_lock)) + return -EIO; + } else { + rte_spinlock_lock(&vf->aq_lock); + } ret = iavf_execute_vf_cmd(adapter, args, async); rte_spinlock_unlock(&vf->aq_lock); -- 2.34.1