From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id B498AA0471 for ; Sat, 22 Jun 2019 15:25:46 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3A3F71C5EF; Sat, 22 Jun 2019 15:25:09 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 658201C5D0 for ; Sat, 22 Jun 2019 15:24:55 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5MDOgcu028293 for ; Sat, 22 Jun 2019 06:24:54 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=GpUD0FWva3/HV9jIOfl/R3efZkT3iYozdFos/qZ8CsU=; b=Kezee6+vQiIJJmV5MMmb/69CDnxCxlD8IG5JDRA6+Ven/AJ9kg7Mu7Pyo/WzAZT2fmdm VexR0ZQvb5cWEsRLp2j1R52umJC4SgIrlMQDvTQRTQwOcoglOtbTTDPctdKxY0rJPKk2 GGOo1lqOyww5hhwIPWkvfWOgjyu0xn/4nPXz1W2+RctPNOm/9qOdMi/hBKVVARkuh6hx a6KU+IgEESOCUeU+H8U5NhUD7ZtYSNOH8zo+rjn78KkxOzAJhrzJIo0/rsQ8vQVDC+EU vO30LFH8VPUl2KjKtAGFelLnOe8kjClDUuu+zpQytBWRcYvxEYLHtmelcfXna45c9pAy 4Q== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0a-0016f401.pphosted.com with ESMTP id 2t9hpnrgf8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 22 Jun 2019 06:24:54 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Sat, 22 Jun 2019 06:24:52 -0700 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Sat, 22 Jun 2019 06:24:52 -0700 Received: from jerin-lab.marvell.com (jerin-lab.marvell.com [10.28.34.14]) by maili.marvell.com (Postfix) with ESMTP id 0828D3F703F; Sat, 22 Jun 2019 06:24:50 -0700 (PDT) From: To: Jerin Jacob , Nithin Dabilpuram , Vamsi Attunuru CC: , Krzysztof Kanas Date: Sat, 22 Jun 2019 18:53:58 +0530 Message-ID: <20190622132417.32694-9-jerinj@marvell.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190622132417.32694-1-jerinj@marvell.com> References: <20190617155537.36144-1-jerinj@marvell.com> <20190622132417.32694-1-jerinj@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-06-22_09:, , signatures=0 Subject: [dpdk-dev] [PATCH v4 08/27] common/octeontx2: introduce irq handling functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Jerin Jacob All PCIe drivers(ethdev, mempool, cryptodev and eventdev) in octeontx2, needs to handle interrupt for mailbox and error handling. Create a helper function over rte interrupt API to register, unregister, disable interrupts. Signed-off-by: Jerin Jacob Signed-off-by: Vamsi Attunuru Signed-off-by: Krzysztof Kanas --- drivers/common/octeontx2/Makefile | 1 + drivers/common/octeontx2/meson.build | 1 + drivers/common/octeontx2/otx2_irq.c | 254 ++++++++++++++++++ drivers/common/octeontx2/otx2_irq.h | 6 + .../rte_common_octeontx2_version.map | 4 + 5 files changed, 266 insertions(+) create mode 100644 drivers/common/octeontx2/otx2_irq.c diff --git a/drivers/common/octeontx2/Makefile b/drivers/common/octeontx2/Makefile index a6f94553d..78243e555 100644 --- a/drivers/common/octeontx2/Makefile +++ b/drivers/common/octeontx2/Makefile @@ -26,6 +26,7 @@ LIBABIVER := 1 # all source are stored in SRCS-y # SRCS-y += otx2_dev.c +SRCS-y += otx2_irq.c SRCS-y += otx2_mbox.c SRCS-y += otx2_common.c diff --git a/drivers/common/octeontx2/meson.build b/drivers/common/octeontx2/meson.build index feaf75d92..44ac90085 100644 --- a/drivers/common/octeontx2/meson.build +++ b/drivers/common/octeontx2/meson.build @@ -3,6 +3,7 @@ # sources= files('otx2_dev.c', + 'otx2_irq.c', 'otx2_mbox.c', 'otx2_common.c', ) diff --git a/drivers/common/octeontx2/otx2_irq.c b/drivers/common/octeontx2/otx2_irq.c new file mode 100644 index 000000000..fa3206af5 --- /dev/null +++ b/drivers/common/octeontx2/otx2_irq.c @@ -0,0 +1,254 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2019 Marvell International Ltd. + */ + +#include +#include +#include +#include + +#include "otx2_common.h" +#include "otx2_irq.h" + +#ifdef RTE_EAL_VFIO + +#include +#include +#include +#include +#include + +#define MAX_INTR_VEC_ID RTE_MAX_RXTX_INTR_VEC_ID +#define MSIX_IRQ_SET_BUF_LEN (sizeof(struct vfio_irq_set) + \ + sizeof(int) * (MAX_INTR_VEC_ID)) + +static int +irq_get_info(struct rte_intr_handle *intr_handle) +{ + struct vfio_irq_info irq = { .argsz = sizeof(irq) }; + int rc; + + irq.index = VFIO_PCI_MSIX_IRQ_INDEX; + + rc = ioctl(intr_handle->vfio_dev_fd, VFIO_DEVICE_GET_IRQ_INFO, &irq); + if (rc < 0) { + otx2_err("Failed to get IRQ info rc=%d errno=%d", rc, errno); + return rc; + } + + otx2_base_dbg("Flags=0x%x index=0x%x count=0x%x max_intr_vec_id=0x%x", + irq.flags, irq.index, irq.count, MAX_INTR_VEC_ID); + + if (irq.count > MAX_INTR_VEC_ID) { + otx2_err("HW max=%d > MAX_INTR_VEC_ID: %d", + intr_handle->max_intr, MAX_INTR_VEC_ID); + intr_handle->max_intr = MAX_INTR_VEC_ID; + } else { + intr_handle->max_intr = irq.count; + } + + return 0; +} + +static int +irq_config(struct rte_intr_handle *intr_handle, unsigned int vec) +{ + char irq_set_buf[MSIX_IRQ_SET_BUF_LEN]; + struct vfio_irq_set *irq_set; + int32_t *fd_ptr; + int len, rc; + + if (vec > intr_handle->max_intr) { + otx2_err("vector=%d greater than max_intr=%d", vec, + intr_handle->max_intr); + return -EINVAL; + } + + len = sizeof(struct vfio_irq_set) + sizeof(int32_t); + + irq_set = (struct vfio_irq_set *)irq_set_buf; + irq_set->argsz = len; + + irq_set->start = vec; + irq_set->count = 1; + irq_set->flags = VFIO_IRQ_SET_DATA_EVENTFD | + VFIO_IRQ_SET_ACTION_TRIGGER; + irq_set->index = VFIO_PCI_MSIX_IRQ_INDEX; + + /* Use vec fd to set interrupt vectors */ + fd_ptr = (int32_t *)&irq_set->data[0]; + fd_ptr[0] = intr_handle->efds[vec]; + + rc = ioctl(intr_handle->vfio_dev_fd, VFIO_DEVICE_SET_IRQS, irq_set); + if (rc) + otx2_err("Failed to set_irqs vector=0x%x rc=%d", vec, rc); + + return rc; +} + +static int +irq_init(struct rte_intr_handle *intr_handle) +{ + char irq_set_buf[MSIX_IRQ_SET_BUF_LEN]; + struct vfio_irq_set *irq_set; + int32_t *fd_ptr; + int len, rc; + uint32_t i; + + if (intr_handle->max_intr > MAX_INTR_VEC_ID) { + otx2_err("Max_intr=%d greater than MAX_INTR_VEC_ID=%d", + intr_handle->max_intr, MAX_INTR_VEC_ID); + return -ERANGE; + } + + len = sizeof(struct vfio_irq_set) + + sizeof(int32_t) * intr_handle->max_intr; + + irq_set = (struct vfio_irq_set *)irq_set_buf; + irq_set->argsz = len; + irq_set->start = 0; + irq_set->count = intr_handle->max_intr; + irq_set->flags = VFIO_IRQ_SET_DATA_EVENTFD | + VFIO_IRQ_SET_ACTION_TRIGGER; + irq_set->index = VFIO_PCI_MSIX_IRQ_INDEX; + + fd_ptr = (int32_t *)&irq_set->data[0]; + for (i = 0; i < irq_set->count; i++) + fd_ptr[i] = -1; + + rc = ioctl(intr_handle->vfio_dev_fd, VFIO_DEVICE_SET_IRQS, irq_set); + if (rc) + otx2_err("Failed to set irqs vector rc=%d", rc); + + return rc; +} + +/** + * @internal + * Disable IRQ + */ +int +otx2_disable_irqs(struct rte_intr_handle *intr_handle) +{ + /* Clear max_intr to indicate re-init next time */ + intr_handle->max_intr = 0; + return rte_intr_disable(intr_handle); +} + +/** + * @internal + * Register IRQ + */ +int +otx2_register_irq(struct rte_intr_handle *intr_handle, + rte_intr_callback_fn cb, void *data, unsigned int vec) +{ + struct rte_intr_handle tmp_handle; + int rc; + + /* If no max_intr read from VFIO */ + if (intr_handle->max_intr == 0) { + irq_get_info(intr_handle); + irq_init(intr_handle); + } + + if (vec > intr_handle->max_intr) { + otx2_err("Vector=%d greater than max_intr=%d", vec, + intr_handle->max_intr); + return -EINVAL; + } + + tmp_handle = *intr_handle; + /* Create new eventfd for interrupt vector */ + tmp_handle.fd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); + if (tmp_handle.fd == -1) + return -ENODEV; + + /* Register vector interrupt callback */ + rc = rte_intr_callback_register(&tmp_handle, cb, data); + if (rc) { + otx2_err("Failed to register vector:0x%x irq callback.", vec); + return rc; + } + + intr_handle->efds[vec] = tmp_handle.fd; + intr_handle->nb_efd = (vec > intr_handle->nb_efd) ? + vec : intr_handle->nb_efd; + if ((intr_handle->nb_efd + 1) > intr_handle->max_intr) + intr_handle->max_intr = intr_handle->nb_efd + 1; + + otx2_base_dbg("Enable vector:0x%x for vfio (efds: %d, max:%d)", + vec, intr_handle->nb_efd, intr_handle->max_intr); + + /* Enable MSIX vectors to VFIO */ + return irq_config(intr_handle, vec); +} + +/** + * @internal + * Unregister IRQ + */ +void +otx2_unregister_irq(struct rte_intr_handle *intr_handle, + rte_intr_callback_fn cb, void *data, unsigned int vec) +{ + struct rte_intr_handle tmp_handle; + + if (vec > intr_handle->max_intr) { + otx2_err("Error unregistering MSI-X interrupts vec:%d > %d", + vec, intr_handle->max_intr); + return; + } + + tmp_handle = *intr_handle; + tmp_handle.fd = intr_handle->efds[vec]; + if (tmp_handle.fd == -1) + return; + + /* Un-register callback func from eal lib */ + rte_intr_callback_unregister(&tmp_handle, cb, data); + + otx2_base_dbg("Disable vector:0x%x for vfio (efds: %d, max:%d)", + vec, intr_handle->nb_efd, intr_handle->max_intr); + + if (intr_handle->efds[vec] != -1) + close(intr_handle->efds[vec]); + /* Disable MSIX vectors from VFIO */ + intr_handle->efds[vec] = -1; + irq_config(intr_handle, vec); +} + +#else + +/** + * @internal + * Register IRQ + */ +int otx2_register_irq(__rte_unused struct rte_intr_handle *intr_handle, + __rte_unused rte_intr_callback_fn cb, + __rte_unused void *data, __rte_unused unsigned int vec) +{ + return -ENOTSUP; +} + + +/** + * @internal + * Unregister IRQ + */ +void otx2_unregister_irq(__rte_unused struct rte_intr_handle *intr_handle, + __rte_unused rte_intr_callback_fn cb, + __rte_unused void *data, __rte_unused unsigned int vec) +{ +} + +/** + * @internal + * Disable IRQ + */ +int otx2_disable_irqs(__rte_unused struct rte_intr_handle *intr_handle) +{ + return -ENOTSUP; +} + +#endif /* RTE_EAL_VFIO */ diff --git a/drivers/common/octeontx2/otx2_irq.h b/drivers/common/octeontx2/otx2_irq.h index df44ddfba..9d326276e 100644 --- a/drivers/common/octeontx2/otx2_irq.h +++ b/drivers/common/octeontx2/otx2_irq.h @@ -16,4 +16,10 @@ typedef struct { uint64_t bits[MAX_VFPF_DWORD_BITS]; } otx2_intr_t; +int otx2_register_irq(struct rte_intr_handle *intr_handle, + rte_intr_callback_fn cb, void *data, unsigned int vec); +void otx2_unregister_irq(struct rte_intr_handle *intr_handle, + rte_intr_callback_fn cb, void *data, unsigned int vec); +int otx2_disable_irqs(struct rte_intr_handle *intr_handle); + #endif /* _OTX2_IRQ_H_ */ diff --git a/drivers/common/octeontx2/rte_common_octeontx2_version.map b/drivers/common/octeontx2/rte_common_octeontx2_version.map index 4d9879899..007649a48 100644 --- a/drivers/common/octeontx2/rte_common_octeontx2_version.map +++ b/drivers/common/octeontx2/rte_common_octeontx2_version.map @@ -21,5 +21,9 @@ DPDK_19.08 { otx2_mbox_msg_send; otx2_mbox_wait_for_rsp; + otx2_disable_irqs; + otx2_unregister_irq; + otx2_register_irq; + local: *; }; -- 2.21.0