From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 691C2A0471 for ; Sat, 22 Jun 2019 15:26:04 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D266A1C601; Sat, 22 Jun 2019 15:25:13 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id E8F201C5B2 for ; Sat, 22 Jun 2019 15:24:59 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5MDOmR8028312 for ; Sat, 22 Jun 2019 06:24:59 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=jJHw1KQ35bQ1ZK0RMpBd6iwgzbFXMjKGLrteQWOtCvI=; b=aVj/I/RqsCRMrVnuIl1FrlvSC1xQcKQteH47gd6RThEwXwG24OV3CdemZCvYcBLanZ5Q YGyrixzCKwZmKxd0tXG6e7oUAJwDp98WtENVdmgdSUIYuH9h/6ELAd+k6nmICv6+fFXe MjLDp3MEKbyNoqCJ7i8EPCres/OIscxTbs7WOOSgwbPlwW7CVmhhcQETibnFBrFUAgJ2 Mb3DZeL7GmHSsa5VE1KLOkdBrwJyPzYoqgEVPnpJZxSNYPSNHcE00PGhT3XGpZ6yuJAb WDMXfE8sL8/aUR5SaK9Cu6Y6atv59t9RTjDvsJRpynrwhQXuGmwx67Pg/uTCrQVZJauo Lw== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0a-0016f401.pphosted.com with ESMTP id 2t9hpnrgfg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 22 Jun 2019 06:24:59 -0700 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sat, 22 Jun 2019 06:24:58 -0700 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Sat, 22 Jun 2019 06:24:57 -0700 Received: from jerin-lab.marvell.com (jerin-lab.marvell.com [10.28.34.14]) by maili.marvell.com (Postfix) with ESMTP id 8236B3F703F; Sat, 22 Jun 2019 06:24:56 -0700 (PDT) From: To: Jerin Jacob , Nithin Dabilpuram , Vamsi Attunuru CC: Date: Sat, 22 Jun 2019 18:54:00 +0530 Message-ID: <20190622132417.32694-11-jerinj@marvell.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190622132417.32694-1-jerinj@marvell.com> References: <20190617155537.36144-1-jerinj@marvell.com> <20190622132417.32694-1-jerinj@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-06-22_09:, , signatures=0 Subject: [dpdk-dev] [PATCH v4 10/27] common/octeontx2: add AF to PF mailbox IRQ and msg handlers X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Nithin Dabilpuram This patch adds support for AF to PF mailbox interrupt and message handling. PF writes the message on mapped mailbox region followed by writing the mailbox doorbell register. Upon receiving, the mailbox request in AF(In Linux kernel), It processes the messages and update the counter memory and update the AF mbox doorbell register. That would trigger a VFIO interrupt to userspace and otx2_process_msgs() will handle it. Signed-off-by: Nithin Dabilpuram Signed-off-by: Vamsi Attunuru --- drivers/common/octeontx2/otx2_dev.c | 120 +++++++++++++++++++++++++++- 1 file changed, 119 insertions(+), 1 deletion(-) diff --git a/drivers/common/octeontx2/otx2_dev.c b/drivers/common/octeontx2/otx2_dev.c index c3b3f9be5..090cfc8f1 100644 --- a/drivers/common/octeontx2/otx2_dev.c +++ b/drivers/common/octeontx2/otx2_dev.c @@ -14,6 +14,9 @@ #include "otx2_dev.h" #include "otx2_mbox.h" +#define RVU_MAX_VF 64 /* RVU_PF_VFPF_MBOX_INT(0..1) */ +#define RVU_MAX_INT_RETRY 3 + /* PF/VF message handling timer */ #define VF_PF_MBOX_TIMER_MS (20 * 1000) @@ -47,6 +50,108 @@ mbox_mem_unmap(void *va, size_t size) munmap(va, size); } +static void +otx2_process_msgs(struct otx2_dev *dev, struct otx2_mbox *mbox) +{ + struct otx2_mbox_dev *mdev = &mbox->dev[0]; + struct mbox_hdr *req_hdr; + struct mbox_msghdr *msg; + int msgs_acked = 0; + int offset; + uint16_t i; + + req_hdr = (struct mbox_hdr *)((uintptr_t)mdev->mbase + mbox->rx_start); + if (req_hdr->num_msgs == 0) + return; + + offset = mbox->rx_start + RTE_ALIGN(sizeof(*req_hdr), MBOX_MSG_ALIGN); + for (i = 0; i < req_hdr->num_msgs; i++) { + msg = (struct mbox_msghdr *)((uintptr_t)mdev->mbase + offset); + + msgs_acked++; + otx2_base_dbg("Message 0x%x (%s) pf:%d/vf:%d", + msg->id, otx2_mbox_id2name(msg->id), + otx2_get_pf(msg->pcifunc), + otx2_get_vf(msg->pcifunc)); + + switch (msg->id) { + /* Add message id's that are handled here */ + case MBOX_MSG_READY: + /* Get our identity */ + dev->pf_func = msg->pcifunc; + break; + + default: + if (msg->rc) + otx2_err("Message (%s) response has err=%d", + otx2_mbox_id2name(msg->id), msg->rc); + break; + } + offset = mbox->rx_start + msg->next_msgoff; + } + + otx2_mbox_reset(mbox, 0); + /* Update acked if someone is waiting a message */ + mdev->msgs_acked = msgs_acked; + rte_wmb(); +} + +static void +otx2_af_pf_mbox_irq(void *param) +{ + struct otx2_dev *dev = param; + uint64_t intr; + + intr = otx2_read64(dev->bar2 + RVU_PF_INT); + if (intr == 0) + return; + + otx2_write64(intr, dev->bar2 + RVU_PF_INT); + + otx2_base_dbg("Irq 0x%" PRIx64 "(pf:%d,vf:%d)", intr, dev->pf, dev->vf); + if (intr) + /* First process all configuration messages */ + otx2_process_msgs(dev, dev->mbox); +} + +static int +mbox_register_irq(struct rte_pci_device *pci_dev, struct otx2_dev *dev) +{ + struct rte_intr_handle *intr_handle = &pci_dev->intr_handle; + int rc; + + otx2_write64(~0ull, dev->bar2 + RVU_PF_INT_ENA_W1C); + + dev->timer_set = 0; + + /* MBOX interrupt AF <-> PF */ + rc = otx2_register_irq(intr_handle, otx2_af_pf_mbox_irq, + dev, RVU_PF_INT_VEC_AFPF_MBOX); + if (rc) { + otx2_err("Fail to register AF<->PF mbox irq"); + return rc; + } + + otx2_write64(~0ull, dev->bar2 + RVU_PF_INT); + otx2_write64(~0ull, dev->bar2 + RVU_PF_INT_ENA_W1S); + + return rc; +} + +static void +mbox_unregister_irq(struct rte_pci_device *pci_dev, struct otx2_dev *dev) +{ + struct rte_intr_handle *intr_handle = &pci_dev->intr_handle; + + otx2_write64(~0ull, dev->bar2 + RVU_PF_INT_ENA_W1C); + + dev->timer_set = 0; + + /* MBOX interrupt AF <-> PF */ + otx2_unregister_irq(intr_handle, otx2_af_pf_mbox_irq, dev, + RVU_PF_INT_VEC_AFPF_MBOX); +} + static void otx2_update_pass_hwcap(struct rte_pci_device *pci_dev, struct otx2_dev *dev) { @@ -120,10 +225,15 @@ otx2_dev_init(struct rte_pci_device *pci_dev, void *otx2_dev) if (rc) goto error; + /* Register mbox interrupts */ + rc = mbox_register_irq(pci_dev, dev); + if (rc) + goto mbox_fini; + /* Check the readiness of PF/VF */ rc = otx2_send_ready_msg(dev->mbox, &dev->pf_func); if (rc) - goto mbox_fini; + goto mbox_unregister; dev->pf = otx2_get_pf(dev->pf_func); dev->vf = otx2_get_vf(dev->pf_func); @@ -162,6 +272,8 @@ otx2_dev_init(struct rte_pci_device *pci_dev, void *otx2_dev) iounmap: mbox_mem_unmap(hwbase, MBOX_SIZE * pci_dev->max_vfs); +mbox_unregister: + mbox_unregister_irq(pci_dev, dev); mbox_fini: otx2_mbox_fini(dev->mbox); otx2_mbox_fini(&dev->mbox_up); @@ -176,6 +288,7 @@ otx2_dev_init(struct rte_pci_device *pci_dev, void *otx2_dev) void otx2_dev_fini(struct rte_pci_device *pci_dev, void *otx2_dev) { + struct rte_intr_handle *intr_handle = &pci_dev->intr_handle; struct otx2_dev *dev = otx2_dev; struct otx2_idev_cfg *idev; struct otx2_mbox *mbox; @@ -185,6 +298,8 @@ otx2_dev_fini(struct rte_pci_device *pci_dev, void *otx2_dev) if (idev->npa_lf && idev->npa_lf->pci_dev == pci_dev) idev->npa_lf = NULL; + mbox_unregister_irq(pci_dev, dev); + /* Release PF - VF */ mbox = &dev->mbox_vfpf; if (mbox->hwbase && mbox->dev) @@ -200,4 +315,7 @@ otx2_dev_fini(struct rte_pci_device *pci_dev, void *otx2_dev) mbox = &dev->mbox_up; otx2_mbox_fini(mbox); dev->mbox_active = 0; + + /* Disable MSIX vectors */ + otx2_disable_irqs(intr_handle); } -- 2.21.0