From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6A10CA0547; Fri, 5 Mar 2021 14:40:11 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6C4C322A357; Fri, 5 Mar 2021 14:39:49 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 2D40E22A348 for ; Fri, 5 Mar 2021 14:39:48 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 125Ddk1C008288 for ; Fri, 5 Mar 2021 05:39:46 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=HaKmufeDpZ4Ei6YCE3vIRoBrFadNZlDnOh8FykjWm1A=; b=PEdM+/UQCYvWkcBe+DN336a98z4QX6oFi5QkLmRgE+fgn+T4EwWX5sABZqhDepYOBA6o uBso6IB7LubgVKnAfqdO3lFJS0Mg1Foig961LSHWJ6MrFWVnS8iAVZHrdAHgx9EGzfg9 wLvbuzRJKFRB6ktX0khzhZh4hXLtepLvPUfMJWyzNQGMuJ9+xTRiJyMAiY4DI8kv+V7J Ka6nratZhhiUhfOX64RXnDMW/5/+7mbSWh7tl/WFNDPD4j5emlzgT8PKuCvFgOYVSKPQ Bdo/3XGEEKxDE30anzcIlg35Oxm0+ieZ56gZjlzBQV0Y0Cy7qHEI+CvmJ+iVF8X1FAto Rw== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 370p7p0daa-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Fri, 05 Mar 2021 05:39:46 -0800 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 5 Mar 2021 05:39:41 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Fri, 5 Mar 2021 05:39:41 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 248293F7041; Fri, 5 Mar 2021 05:39:38 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , Date: Fri, 5 Mar 2021 19:08:30 +0530 Message-ID: <20210305133918.8005-5-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210305133918.8005-1-ndabilpuram@marvell.com> References: <20210305133918.8005-1-ndabilpuram@marvell.com> MIME-Version: 1.0 Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-05_08:2021-03-03, 2021-03-05 signatures=0 Subject: [dpdk-dev] [PATCH 04/52] common/cnxk: add interrupt helper API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Jerin Jacob Add interrupt helper API's in common code to register and unregister for specific interrupt vectors. These API's will be used by all cnxk drivers. Signed-off-by: Jerin Jacob --- drivers/common/cnxk/meson.build | 3 +- drivers/common/cnxk/roc_dev_priv.h | 14 +++ drivers/common/cnxk/roc_irq.c | 249 +++++++++++++++++++++++++++++++++++++ drivers/common/cnxk/roc_platform.c | 4 + drivers/common/cnxk/roc_platform.h | 11 ++ drivers/common/cnxk/roc_priv.h | 3 + drivers/common/cnxk/version.map | 1 + 7 files changed, 284 insertions(+), 1 deletion(-) create mode 100644 drivers/common/cnxk/roc_dev_priv.h create mode 100644 drivers/common/cnxk/roc_irq.c diff --git a/drivers/common/cnxk/meson.build b/drivers/common/cnxk/meson.build index e5de6cf..e2a2da2 100644 --- a/drivers/common/cnxk/meson.build +++ b/drivers/common/cnxk/meson.build @@ -11,7 +11,8 @@ endif config_flag_fmt = 'RTE_LIBRTE_@0@_COMMON' deps = ['eal', 'pci', 'bus_pci', 'mbuf'] -sources = files('roc_model.c', +sources = files('roc_irq.c', + 'roc_model.c', 'roc_platform.c', 'roc_utils.c') includes += include_directories('../../bus/pci') diff --git a/drivers/common/cnxk/roc_dev_priv.h b/drivers/common/cnxk/roc_dev_priv.h new file mode 100644 index 0000000..836e75b --- /dev/null +++ b/drivers/common/cnxk/roc_dev_priv.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2020 Marvell. + */ + +#ifndef _ROC_DEV_PRIV_H +#define _ROC_DEV_PRIV_H + +int dev_irq_register(struct plt_intr_handle *intr_handle, + plt_intr_callback_fn cb, void *data, unsigned int vec); +void dev_irq_unregister(struct plt_intr_handle *intr_handle, + plt_intr_callback_fn cb, void *data, unsigned int vec); +int dev_irqs_disable(struct plt_intr_handle *intr_handle); + +#endif /* _ROC_DEV_PRIV_H */ diff --git a/drivers/common/cnxk/roc_irq.c b/drivers/common/cnxk/roc_irq.c new file mode 100644 index 0000000..75d8d7a --- /dev/null +++ b/drivers/common/cnxk/roc_irq.c @@ -0,0 +1,249 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2020 Marvell. + */ + +#include "roc_api.h" +#include "roc_priv.h" + +#if defined(__linux__) + +#include +#include +#include +#include +#include + +#define MSIX_IRQ_SET_BUF_LEN \ + (sizeof(struct vfio_irq_set) + sizeof(int) * (PLT_MAX_RXTX_INTR_VEC_ID)) + +static int +irq_get_info(struct plt_intr_handle *intr_handle) +{ + struct vfio_irq_info irq = {.argsz = sizeof(irq)}; + int rc; + + irq.index = VFIO_PCI_MSIX_IRQ_INDEX; + + rc = ioctl(intr_handle->vfio_dev_fd, VFIO_DEVICE_GET_IRQ_INFO, &irq); + if (rc < 0) { + plt_err("Failed to get IRQ info rc=%d errno=%d", rc, errno); + return rc; + } + + plt_base_dbg("Flags=0x%x index=0x%x count=0x%x max_intr_vec_id=0x%x", + irq.flags, irq.index, irq.count, PLT_MAX_RXTX_INTR_VEC_ID); + + if (irq.count > PLT_MAX_RXTX_INTR_VEC_ID) { + plt_err("HW max=%d > PLT_MAX_RXTX_INTR_VEC_ID: %d", irq.count, + PLT_MAX_RXTX_INTR_VEC_ID); + intr_handle->max_intr = PLT_MAX_RXTX_INTR_VEC_ID; + } else { + intr_handle->max_intr = irq.count; + } + + return 0; +} + +static int +irq_config(struct plt_intr_handle *intr_handle, unsigned int vec) +{ + char irq_set_buf[MSIX_IRQ_SET_BUF_LEN]; + struct vfio_irq_set *irq_set; + int32_t *fd_ptr; + int len, rc; + + if (vec > intr_handle->max_intr) { + plt_err("vector=%d greater than max_intr=%d", vec, + intr_handle->max_intr); + return -EINVAL; + } + + len = sizeof(struct vfio_irq_set) + sizeof(int32_t); + + irq_set = (struct vfio_irq_set *)irq_set_buf; + irq_set->argsz = len; + + irq_set->start = vec; + irq_set->count = 1; + irq_set->flags = + VFIO_IRQ_SET_DATA_EVENTFD | VFIO_IRQ_SET_ACTION_TRIGGER; + irq_set->index = VFIO_PCI_MSIX_IRQ_INDEX; + + /* Use vec fd to set interrupt vectors */ + fd_ptr = (int32_t *)&irq_set->data[0]; + fd_ptr[0] = intr_handle->efds[vec]; + + rc = ioctl(intr_handle->vfio_dev_fd, VFIO_DEVICE_SET_IRQS, irq_set); + if (rc) + plt_err("Failed to set_irqs vector=0x%x rc=%d", vec, rc); + + return rc; +} + +static int +irq_init(struct plt_intr_handle *intr_handle) +{ + char irq_set_buf[MSIX_IRQ_SET_BUF_LEN]; + struct vfio_irq_set *irq_set; + int32_t *fd_ptr; + int len, rc; + uint32_t i; + + if (intr_handle->max_intr > PLT_MAX_RXTX_INTR_VEC_ID) { + plt_err("Max_intr=%d greater than PLT_MAX_RXTX_INTR_VEC_ID=%d", + intr_handle->max_intr, PLT_MAX_RXTX_INTR_VEC_ID); + return -ERANGE; + } + + len = sizeof(struct vfio_irq_set) + + sizeof(int32_t) * intr_handle->max_intr; + + irq_set = (struct vfio_irq_set *)irq_set_buf; + irq_set->argsz = len; + irq_set->start = 0; + irq_set->count = intr_handle->max_intr; + irq_set->flags = + VFIO_IRQ_SET_DATA_EVENTFD | VFIO_IRQ_SET_ACTION_TRIGGER; + irq_set->index = VFIO_PCI_MSIX_IRQ_INDEX; + + fd_ptr = (int32_t *)&irq_set->data[0]; + for (i = 0; i < irq_set->count; i++) + fd_ptr[i] = -1; + + rc = ioctl(intr_handle->vfio_dev_fd, VFIO_DEVICE_SET_IRQS, irq_set); + if (rc) + plt_err("Failed to set irqs vector rc=%d", rc); + + return rc; +} + +int +dev_irqs_disable(struct plt_intr_handle *intr_handle) +{ + /* Clear max_intr to indicate re-init next time */ + intr_handle->max_intr = 0; + return plt_intr_disable(intr_handle); +} + +int +dev_irq_register(struct plt_intr_handle *intr_handle, plt_intr_callback_fn cb, + void *data, unsigned int vec) +{ + struct plt_intr_handle tmp_handle; + int rc; + + /* If no max_intr read from VFIO */ + if (intr_handle->max_intr == 0) { + irq_get_info(intr_handle); + irq_init(intr_handle); + } + + if (vec > intr_handle->max_intr) { + plt_err("Vector=%d greater than max_intr=%d", vec, + intr_handle->max_intr); + return -EINVAL; + } + + tmp_handle = *intr_handle; + /* Create new eventfd for interrupt vector */ + tmp_handle.fd = eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); + if (tmp_handle.fd == -1) + return -ENODEV; + + /* Register vector interrupt callback */ + rc = plt_intr_callback_register(&tmp_handle, cb, data); + if (rc) { + plt_err("Failed to register vector:0x%x irq callback.", vec); + return rc; + } + + intr_handle->efds[vec] = tmp_handle.fd; + intr_handle->nb_efd = + (vec > intr_handle->nb_efd) ? vec : intr_handle->nb_efd; + if ((intr_handle->nb_efd + 1) > intr_handle->max_intr) + intr_handle->max_intr = intr_handle->nb_efd + 1; + + plt_base_dbg("Enable vector:0x%x for vfio (efds: %d, max:%d)", vec, + intr_handle->nb_efd, intr_handle->max_intr); + + /* Enable MSIX vectors to VFIO */ + return irq_config(intr_handle, vec); +} + +void +dev_irq_unregister(struct plt_intr_handle *intr_handle, plt_intr_callback_fn cb, + void *data, unsigned int vec) +{ + struct plt_intr_handle tmp_handle; + uint8_t retries = 5; /* 5 ms */ + int rc; + + if (vec > intr_handle->max_intr) { + plt_err("Error unregistering MSI-X interrupts vec:%d > %d", vec, + intr_handle->max_intr); + return; + } + + tmp_handle = *intr_handle; + tmp_handle.fd = intr_handle->efds[vec]; + if (tmp_handle.fd == -1) + return; + + do { + /* Un-register callback func from platform lib */ + rc = plt_intr_callback_unregister(&tmp_handle, cb, data); + /* Retry only if -EAGAIN */ + if (rc != -EAGAIN) + break; + plt_delay_ms(1); + retries--; + } while (retries); + + if (rc < 0) { + plt_err("Error unregistering MSI-X vec %d cb, rc=%d", vec, rc); + return; + } + + plt_base_dbg("Disable vector:0x%x for vfio (efds: %d, max:%d)", vec, + intr_handle->nb_efd, intr_handle->max_intr); + + if (intr_handle->efds[vec] != -1) + close(intr_handle->efds[vec]); + /* Disable MSIX vectors from VFIO */ + intr_handle->efds[vec] = -1; + irq_config(intr_handle, vec); +} + +#else + +int +dev_irq_register(struct plt_intr_handle *intr_handle, plt_intr_callback_fn cb, + void *data, unsigned int vec) +{ + PLT_SET_USED(intr_handle); + PLT_SET_USED(cb); + PLT_SET_USED(data); + PLT_SET_USED(vec); + + return -ENOTSUP; +} + +void +dev_irq_unregister(struct plt_intr_handle *intr_handle, plt_intr_callback_fn cb, + void *data, unsigned int vec) +{ + PLT_SET_USED(intr_handle); + PLT_SET_USED(cb); + PLT_SET_USED(data); + PLT_SET_USED(vec); +} + +int +dev_irqs_disable(struct plt_intr_handle *intr_handle) +{ + PLT_SET_USED(intr_handle); + + return -ENOTSUP; +} + +#endif /* __linux__ */ diff --git a/drivers/common/cnxk/roc_platform.c b/drivers/common/cnxk/roc_platform.c index 072b3e5..79e9171 100644 --- a/drivers/common/cnxk/roc_platform.c +++ b/drivers/common/cnxk/roc_platform.c @@ -2,6 +2,8 @@ * Copyright(C) 2019 Marvell International Ltd. */ +#include + #include "roc_api.h" int @@ -24,3 +26,5 @@ plt_init(void) roc_model_init(mz->addr); return 0; } + +RTE_LOG_REGISTER(cnxk_logtype_base, pmd.cnxk.base, NOTICE); diff --git a/drivers/common/cnxk/roc_platform.h b/drivers/common/cnxk/roc_platform.h index 7477613..ea38216 100644 --- a/drivers/common/cnxk/roc_platform.h +++ b/drivers/common/cnxk/roc_platform.h @@ -124,12 +124,23 @@ #define plt_strlcpy rte_strlcpy /* Log */ +extern int cnxk_logtype_base; #define plt_err(fmt, args...) \ RTE_LOG(ERR, PMD, "%s():%u " fmt "\n", __func__, __LINE__, ##args) #define plt_info(fmt, args...) RTE_LOG(INFO, PMD, fmt "\n", ##args) #define plt_warn(fmt, args...) RTE_LOG(WARNING, PMD, fmt "\n", ##args) #define plt_print(fmt, args...) RTE_LOG(INFO, PMD, fmt "\n", ##args) +/** + * Log debug message if given subsystem logging is enabled. + */ +#define plt_dbg(subsystem, fmt, args...) \ + rte_log(RTE_LOG_DEBUG, cnxk_logtype_##subsystem, \ + "[%s] %s():%u " fmt "\n", #subsystem, __func__, __LINE__, \ + ##args) + +#define plt_base_dbg(fmt, ...) plt_dbg(base, fmt, ##__VA_ARGS__) + #ifdef __cplusplus #define CNXK_PCI_ID(subsystem_dev, dev) \ { \ diff --git a/drivers/common/cnxk/roc_priv.h b/drivers/common/cnxk/roc_priv.h index be37c6c..8c250f4 100644 --- a/drivers/common/cnxk/roc_priv.h +++ b/drivers/common/cnxk/roc_priv.h @@ -8,4 +8,7 @@ /* Utils */ #include "roc_util_priv.h" +/* Dev */ +#include "roc_dev_priv.h" + #endif /* _ROC_PRIV_H_ */ diff --git a/drivers/common/cnxk/version.map b/drivers/common/cnxk/version.map index 227f2ce..e59bde9 100644 --- a/drivers/common/cnxk/version.map +++ b/drivers/common/cnxk/version.map @@ -1,6 +1,7 @@ INTERNAL { global: + cnxk_logtype_base; plt_init; roc_error_msg_get; roc_model; -- 2.8.4