From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 18725A0471 for ; Mon, 17 Jun 2019 17:58:44 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2B18E1BF92; Mon, 17 Jun 2019 17:56:50 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id E76F81BF7E for ; Mon, 17 Jun 2019 17:56:37 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5HFppxP000998 for ; Mon, 17 Jun 2019 08:56:37 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=n+h2SslRYMl3qCsU4PL1rp+RsLFXMRd+GRNW10MFTUg=; b=iQz4k7v1JNQC+cIMBof0L0meEJyaf1w4Sbc7Z5Us3LBThD9VTKnT3/C6oyNrQhiXxbkJ Nb0bt2fXs89AaZrtGkdyTnzswVnY2C9qGJqk9PbHq9X09YI2xA86NCTm41HXyY40cu7w gn5v03xlmnsWStuWRvtiTkdv6rLYmewKyWz9Imx7sF6Y7o/gDOKYDyARhSsoHh/QDaIM pE7UxeunphmygHgI2fdIt1WOQmmbNeMjvxq0Il+VIgm9RQT5YDWNYrJ/4lJQM9zH5Qrp OzvrtSuYJAQ1qwObwoH8+lxtv00dh2KCXGULZuYOctG5dsII9saRTPSO+8dqSaRG1b8C 7g== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0b-0016f401.pphosted.com with ESMTP id 2t506hyax9-15 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 17 Jun 2019 08:56:37 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Mon, 17 Jun 2019 08:56:31 -0700 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Mon, 17 Jun 2019 08:56:31 -0700 Received: from jerin-lab.marvell.com (jerin-lab.marvell.com [10.28.34.14]) by maili.marvell.com (Postfix) with ESMTP id A9A773F704A; Mon, 17 Jun 2019 08:56:29 -0700 (PDT) From: To: , Jerin Jacob , Nithin Dabilpuram , Vamsi Attunuru CC: Harman Kalra Date: Mon, 17 Jun 2019 21:25:24 +0530 Message-ID: <20190617155537.36144-15-jerinj@marvell.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190617155537.36144-1-jerinj@marvell.com> References: <20190601014905.45531-1-jerinj@marvell.com> <20190617155537.36144-1-jerinj@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-06-17_07:, , signatures=0 Subject: [dpdk-dev] [PATCH v3 14/27] common/octeontx2: add FLR IRQ handler X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Nithin Dabilpuram Upon receiving FLR request from VF, It is PF responsibly forward to AF and enable FLR for VFs. This patch adds support for VF FLR support in PF. This patch also add otx2_dev_active_vfs() API to find the number of active VF for given PF. Signed-off-by: Nithin Dabilpuram Signed-off-by: Harman Kalra --- drivers/common/octeontx2/otx2_dev.c | 180 ++++++++++++++++++ .../rte_common_octeontx2_version.map | 1 + 2 files changed, 181 insertions(+) diff --git a/drivers/common/octeontx2/otx2_dev.c b/drivers/common/octeontx2/otx2_dev.c index 09943855d..53a0c6efb 100644 --- a/drivers/common/octeontx2/otx2_dev.c +++ b/drivers/common/octeontx2/otx2_dev.c @@ -51,6 +51,52 @@ mbox_mem_unmap(void *va, size_t size) munmap(va, size); } +static int +pf_af_sync_msg(struct otx2_dev *dev, struct mbox_msghdr **rsp) +{ + uint32_t timeout = 0, sleep = 1; struct otx2_mbox *mbox = dev->mbox; + struct otx2_mbox_dev *mdev = &mbox->dev[0]; + volatile uint64_t int_status; + struct mbox_msghdr *msghdr; + uint64_t off; + int rc = 0; + + /* We need to disable PF interrupts. We are in timer interrupt */ + otx2_write64(~0ull, dev->bar2 + RVU_PF_INT_ENA_W1C); + + /* Send message */ + otx2_mbox_msg_send(mbox, 0); + + do { + rte_delay_ms(sleep); + timeout += sleep; + if (timeout >= MBOX_RSP_TIMEOUT) { + otx2_err("Message timeout: %dms", MBOX_RSP_TIMEOUT); + rc = -EIO; + break; + } + int_status = otx2_read64(dev->bar2 + RVU_PF_INT); + } while ((int_status & 0x1) != 0x1); + + /* Clear */ + otx2_write64(int_status, dev->bar2 + RVU_PF_INT); + + /* Enable interrupts */ + otx2_write64(~0ull, dev->bar2 + RVU_PF_INT_ENA_W1S); + + if (rc == 0) { + /* Get message */ + off = mbox->rx_start + + RTE_ALIGN(sizeof(struct mbox_hdr), MBOX_MSG_ALIGN); + msghdr = (struct mbox_msghdr *)((uintptr_t)mdev->mbase + off); + if (rsp) + *rsp = msghdr; + rc = msghdr->rc; + } + + return rc; +} + static int af_pf_wait_msg(struct otx2_dev *dev, uint16_t vf, int num_msg) { @@ -703,6 +749,132 @@ mbox_unregister_irq(struct rte_pci_device *pci_dev, struct otx2_dev *dev) return mbox_unregister_pf_irq(pci_dev, dev); } +static int +vf_flr_send_msg(struct otx2_dev *dev, uint16_t vf) +{ + struct otx2_mbox *mbox = dev->mbox; + struct msg_req *req; + int rc; + + req = otx2_mbox_alloc_msg_vf_flr(mbox); + /* Overwrite pcifunc to indicate VF */ + req->hdr.pcifunc = otx2_pfvf_func(dev->pf, vf); + + /* Sync message in interrupt context */ + rc = pf_af_sync_msg(dev, NULL); + if (rc) + otx2_err("Failed to send VF FLR mbox msg, rc=%d", rc); + + return rc; +} + +static void +otx2_pf_vf_flr_irq(void *param) +{ + struct otx2_dev *dev = (struct otx2_dev *)param; + uint16_t max_vf = 64, vf; + uintptr_t bar2; + uint64_t intr; + int i; + + max_vf = (dev->maxvf > 0) ? dev->maxvf : 64; + bar2 = dev->bar2; + + otx2_base_dbg("FLR VF interrupt: max_vf: %d", max_vf); + + for (i = 0; i < MAX_VFPF_DWORD_BITS; ++i) { + intr = otx2_read64(bar2 + RVU_PF_VFFLR_INTX(i)); + if (!intr) + continue; + + for (vf = 0; vf < max_vf; vf++) { + if (!(intr & (1ULL << vf))) + continue; + + vf = 64 * i + vf; + otx2_base_dbg("FLR: i :%d intr: 0x%" PRIx64 ", vf-%d", + i, intr, vf); + /* Clear interrupt */ + otx2_write64(BIT_ULL(vf), bar2 + RVU_PF_VFFLR_INTX(i)); + /* Disable the interrupt */ + otx2_write64(BIT_ULL(vf), + bar2 + RVU_PF_VFFLR_INT_ENA_W1CX(i)); + /* Inform AF about VF reset */ + vf_flr_send_msg(dev, vf); + + /* Signal FLR finish */ + otx2_write64(BIT_ULL(vf), bar2 + RVU_PF_VFTRPENDX(i)); + /* Enable interrupt */ + otx2_write64(~0ull, + bar2 + RVU_PF_VFFLR_INT_ENA_W1SX(i)); + } + } +} + +static int +vf_flr_unregister_irqs(struct rte_pci_device *pci_dev, struct otx2_dev *dev) +{ + struct rte_intr_handle *intr_handle = &pci_dev->intr_handle; + int i; + + otx2_base_dbg("Unregister VF FLR interrupts for %s", pci_dev->name); + + /* HW clear irq */ + for (i = 0; i < MAX_VFPF_DWORD_BITS; i++) + otx2_write64(~0ull, dev->bar2 + RVU_PF_VFFLR_INT_ENA_W1CX(i)); + + otx2_unregister_irq(intr_handle, otx2_pf_vf_flr_irq, dev, + RVU_PF_INT_VEC_VFFLR0); + + otx2_unregister_irq(intr_handle, otx2_pf_vf_flr_irq, dev, + RVU_PF_INT_VEC_VFFLR1); + + return 0; +} + +static int +vf_flr_register_irqs(struct rte_pci_device *pci_dev, struct otx2_dev *dev) +{ + struct rte_intr_handle *handle = &pci_dev->intr_handle; + int i, rc; + + otx2_base_dbg("Register VF FLR interrupts for %s", pci_dev->name); + + rc = otx2_register_irq(handle, otx2_pf_vf_flr_irq, dev, + RVU_PF_INT_VEC_VFFLR0); + if (rc) + otx2_err("Failed to init RVU_PF_INT_VEC_VFFLR0 rc=%d", rc); + + rc = otx2_register_irq(handle, otx2_pf_vf_flr_irq, dev, + RVU_PF_INT_VEC_VFFLR1); + if (rc) + otx2_err("Failed to init RVU_PF_INT_VEC_VFFLR1 rc=%d", rc); + + /* Enable HW interrupt */ + for (i = 0; i < MAX_VFPF_DWORD_BITS; ++i) { + otx2_write64(~0ull, dev->bar2 + RVU_PF_VFFLR_INTX(i)); + otx2_write64(~0ull, dev->bar2 + RVU_PF_VFTRPENDX(i)); + otx2_write64(~0ull, dev->bar2 + RVU_PF_VFFLR_INT_ENA_W1SX(i)); + } + return 0; +} + +/** + * @internal + * Get number of active VFs for the given PF device. + */ +int +otx2_dev_active_vfs(void *otx2_dev) +{ + struct otx2_dev *dev = otx2_dev; + int i, count = 0; + + for (i = 0; i < MAX_VFPF_DWORD_BITS; i++) + count += __builtin_popcount(dev->active_vfs[i]); + + return count; +} + static void otx2_update_pass_hwcap(struct rte_pci_device *pci_dev, struct otx2_dev *dev) { @@ -818,6 +990,12 @@ otx2_dev_init(struct rte_pci_device *pci_dev, void *otx2_dev) goto mbox_fini; } + /* Register VF-FLR irq handlers */ + if (otx2_dev_is_pf(dev)) { + rc = vf_flr_register_irqs(pci_dev, dev); + if (rc) + goto iounmap; + } dev->mbox_active = 1; return rc; @@ -851,6 +1029,8 @@ otx2_dev_fini(struct rte_pci_device *pci_dev, void *otx2_dev) mbox_unregister_irq(pci_dev, dev); + if (otx2_dev_is_pf(dev)) + vf_flr_unregister_irqs(pci_dev, dev); /* Release PF - VF */ mbox = &dev->mbox_vfpf; if (mbox->hwbase && mbox->dev) diff --git a/drivers/common/octeontx2/rte_common_octeontx2_version.map b/drivers/common/octeontx2/rte_common_octeontx2_version.map index efcf0cb55..2f4826311 100644 --- a/drivers/common/octeontx2/rte_common_octeontx2_version.map +++ b/drivers/common/octeontx2/rte_common_octeontx2_version.map @@ -1,6 +1,7 @@ DPDK_19.08 { global: + otx2_dev_active_vfs; otx2_dev_fini; otx2_dev_init; -- 2.21.0