From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 83B42A0C4E; Mon, 25 Oct 2021 16:27:43 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 84A8E41144; Mon, 25 Oct 2021 16:27:38 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mails.dpdk.org (Postfix) with ESMTP id 961404111B for ; Mon, 25 Oct 2021 16:27:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1635172057; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iK0/pkOOvyi1b+S+aC32WHgBnl0bCutAFV5ezJ82eJw=; b=IOImdZIk0e2XrqUdB8Rbd+HLXV+efYAg3lWfQLL9AsbJhecKV/IKB83LWwks4K7dEC7zCy IYc5L1LR51NWv6xSVLt9EjXeNBBgTk7y4u4AmlCGQRjEq3+jauY/xbtQyPuNNqCddOI2Sa 8dZRpggTiRt0gi1oE0IJOgMDfyCyMOg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-531-1wKiWaKPN_qYplaFXg81pg-1; Mon, 25 Oct 2021 10:27:33 -0400 X-MC-Unique: 1wKiWaKPN_qYplaFXg81pg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C3EAA18125C2; Mon, 25 Oct 2021 14:27:31 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.192.64]) by smtp.corp.redhat.com (Postfix) with ESMTP id A037C19D9D; Mon, 25 Oct 2021 14:27:28 +0000 (UTC) From: David Marchand To: hkalra@marvell.com, dev@dpdk.org Cc: dmitry.kozliuk@gmail.com, rasland@nvidia.com, thomas@monjalon.net, Ray Kinsella Date: Mon, 25 Oct 2021 16:27:04 +0200 Message-Id: <20211025142712.1273-2-david.marchand@redhat.com> In-Reply-To: <20211025142712.1273-1-david.marchand@redhat.com> References: <20211022204934.132186-1-hkalra@marvell.com> <20211025142712.1273-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="US-ASCII" Subject: [dpdk-dev] [PATCH v8 1/9] interrupts: add allocator and accessors X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Harman Kalra Prototype/Implement get set APIs for interrupt handle fields. User won't be able to access any of the interrupt handle fields directly while should use these get/set APIs to access/manipulate them. Internal interrupt header i.e. rte_eal_interrupt.h is rearranged, as APIs defined are moved to rte_interrupts.h and epoll specific definitions are moved to a new header rte_epoll.h. Later in the series rte_eal_interrupt.h will be removed. Signed-off-by: Harman Kalra Acked-by: Ray Kinsella Acked-by: Dmitry Kozlyuk Signed-off-by: David Marchand --- Changes since v7: - lowered checks log level to DEBUG, - removed asserts on vector list size, and fixed check on list size for drivers like mlx5 who expects list is not initialized, Changes since v5: - renamed RTE_INTR_INSTANCE_F_UNSHARED as RTE_INTR_INSTANCE_F_PRIVATE, - used a single bit to mark instance as shared (default is private), - removed rte_intr_instance_copy / rte_intr_instance_alloc_flag_get with a single rte_intr_instance_dup helper, - made rte_intr_vec_list_alloc alloc_flags-aware, - exported all symbols for Windows, --- MAINTAINERS | 1 + lib/eal/common/eal_common_interrupts.c | 407 ++++++++++++++++ lib/eal/common/meson.build | 1 + lib/eal/include/meson.build | 1 + lib/eal/include/rte_eal_interrupts.h | 207 +------- lib/eal/include/rte_epoll.h | 118 +++++ lib/eal/include/rte_interrupts.h | 627 +++++++++++++++++++++++++ lib/eal/version.map | 45 +- 8 files changed, 1197 insertions(+), 210 deletions(-) create mode 100644 lib/eal/common/eal_common_interrupts.c create mode 100644 lib/eal/include/rte_epoll.h diff --git a/MAINTAINERS b/MAINTAINERS index 587632dce0..097a57f7f6 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -211,6 +211,7 @@ F: app/test/test_memzone.c Interrupt Subsystem M: Harman Kalra +F: lib/eal/include/rte_epoll.h F: lib/eal/*/*interrupts.* F: app/test/test_interrupts.c diff --git a/lib/eal/common/eal_common_interrupts.c b/lib/eal/common/eal_common_interrupts.c new file mode 100644 index 0000000000..46064870f4 --- /dev/null +++ b/lib/eal/common/eal_common_interrupts.c @@ -0,0 +1,407 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include +#include + +#include +#include +#include +#include + +/* Macros to check for valid interrupt handle */ +#define CHECK_VALID_INTR_HANDLE(intr_handle) do { \ + if (intr_handle == NULL) { \ + RTE_LOG(DEBUG, EAL, "Interrupt instance unallocated\n"); \ + rte_errno = EINVAL; \ + goto fail; \ + } \ +} while (0) + +#define RTE_INTR_INSTANCE_KNOWN_FLAGS (RTE_INTR_INSTANCE_F_PRIVATE \ + | RTE_INTR_INSTANCE_F_SHARED \ + ) + +#define RTE_INTR_INSTANCE_USES_RTE_MEMORY(flags) \ + !!(flags & RTE_INTR_INSTANCE_F_SHARED) + +struct rte_intr_handle *rte_intr_instance_alloc(uint32_t flags) +{ + struct rte_intr_handle *intr_handle; + bool uses_rte_memory; + + /* Check the flag passed by user, it should be part of the + * defined flags. + */ + if ((flags & ~RTE_INTR_INSTANCE_KNOWN_FLAGS) != 0) { + RTE_LOG(DEBUG, EAL, "Invalid alloc flag passed 0x%x\n", flags); + rte_errno = EINVAL; + return NULL; + } + + uses_rte_memory = RTE_INTR_INSTANCE_USES_RTE_MEMORY(flags); + if (uses_rte_memory) + intr_handle = rte_zmalloc(NULL, sizeof(*intr_handle), 0); + else + intr_handle = calloc(1, sizeof(*intr_handle)); + if (intr_handle == NULL) { + RTE_LOG(ERR, EAL, "Failed to allocate intr_handle\n"); + rte_errno = ENOMEM; + return NULL; + } + + intr_handle->alloc_flags = flags; + intr_handle->nb_intr = RTE_MAX_RXTX_INTR_VEC_ID; + + return intr_handle; +} + +struct rte_intr_handle *rte_intr_instance_dup(const struct rte_intr_handle *src) +{ + struct rte_intr_handle *intr_handle; + + if (src == NULL) { + RTE_LOG(DEBUG, EAL, "Source interrupt instance unallocated\n"); + rte_errno = EINVAL; + return NULL; + } + + intr_handle = rte_intr_instance_alloc(src->alloc_flags); + + intr_handle->fd = src->fd; + intr_handle->vfio_dev_fd = src->vfio_dev_fd; + intr_handle->type = src->type; + intr_handle->max_intr = src->max_intr; + intr_handle->nb_efd = src->nb_efd; + intr_handle->efd_counter_size = src->efd_counter_size; + memcpy(intr_handle->efds, src->efds, src->nb_intr); + memcpy(intr_handle->elist, src->elist, src->nb_intr); + + return intr_handle; +} + +void rte_intr_instance_free(struct rte_intr_handle *intr_handle) +{ + if (intr_handle == NULL) + return; + if (RTE_INTR_INSTANCE_USES_RTE_MEMORY(intr_handle->alloc_flags)) + rte_free(intr_handle); + else + free(intr_handle); +} + +int rte_intr_fd_set(struct rte_intr_handle *intr_handle, int fd) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + intr_handle->fd = fd; + + return 0; +fail: + return -rte_errno; +} + +int rte_intr_fd_get(const struct rte_intr_handle *intr_handle) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + return intr_handle->fd; +fail: + return -1; +} + +int rte_intr_type_set(struct rte_intr_handle *intr_handle, + enum rte_intr_handle_type type) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + intr_handle->type = type; + + return 0; +fail: + return -rte_errno; +} + +enum rte_intr_handle_type rte_intr_type_get( + const struct rte_intr_handle *intr_handle) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + return intr_handle->type; +fail: + return RTE_INTR_HANDLE_UNKNOWN; +} + +int rte_intr_dev_fd_set(struct rte_intr_handle *intr_handle, int fd) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + intr_handle->vfio_dev_fd = fd; + + return 0; +fail: + return -rte_errno; +} + +int rte_intr_dev_fd_get(const struct rte_intr_handle *intr_handle) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + return intr_handle->vfio_dev_fd; +fail: + return -1; +} + +int rte_intr_max_intr_set(struct rte_intr_handle *intr_handle, + int max_intr) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + if (max_intr > intr_handle->nb_intr) { + RTE_LOG(DEBUG, EAL, "Maximum interrupt vector ID (%d) exceeds " + "the number of available events (%d)\n", max_intr, + intr_handle->nb_intr); + rte_errno = ERANGE; + goto fail; + } + + intr_handle->max_intr = max_intr; + + return 0; +fail: + return -rte_errno; +} + +int rte_intr_max_intr_get(const struct rte_intr_handle *intr_handle) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + return intr_handle->max_intr; +fail: + return -rte_errno; +} + +int rte_intr_nb_efd_set(struct rte_intr_handle *intr_handle, int nb_efd) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + intr_handle->nb_efd = nb_efd; + + return 0; +fail: + return -rte_errno; +} + +int rte_intr_nb_efd_get(const struct rte_intr_handle *intr_handle) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + return intr_handle->nb_efd; +fail: + return -rte_errno; +} + +int rte_intr_nb_intr_get(const struct rte_intr_handle *intr_handle) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + return intr_handle->nb_intr; +fail: + return -rte_errno; +} + +int rte_intr_efd_counter_size_set(struct rte_intr_handle *intr_handle, + uint8_t efd_counter_size) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + intr_handle->efd_counter_size = efd_counter_size; + + return 0; +fail: + return -rte_errno; +} + +int rte_intr_efd_counter_size_get(const struct rte_intr_handle *intr_handle) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + return intr_handle->efd_counter_size; +fail: + return -rte_errno; +} + +int rte_intr_efds_index_get(const struct rte_intr_handle *intr_handle, + int index) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + if (index >= intr_handle->nb_intr) { + RTE_LOG(DEBUG, EAL, "Invalid index %d, max limit %d\n", index, + intr_handle->nb_intr); + rte_errno = EINVAL; + goto fail; + } + + return intr_handle->efds[index]; +fail: + return -rte_errno; +} + +int rte_intr_efds_index_set(struct rte_intr_handle *intr_handle, + int index, int fd) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + if (index >= intr_handle->nb_intr) { + RTE_LOG(DEBUG, EAL, "Invalid index %d, max limit %d\n", index, + intr_handle->nb_intr); + rte_errno = ERANGE; + goto fail; + } + + intr_handle->efds[index] = fd; + + return 0; +fail: + return -rte_errno; +} + +struct rte_epoll_event *rte_intr_elist_index_get( + struct rte_intr_handle *intr_handle, int index) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + if (index >= intr_handle->nb_intr) { + RTE_LOG(DEBUG, EAL, "Invalid index %d, max limit %d\n", index, + intr_handle->nb_intr); + rte_errno = ERANGE; + goto fail; + } + + return &intr_handle->elist[index]; +fail: + return NULL; +} + +int rte_intr_elist_index_set(struct rte_intr_handle *intr_handle, + int index, struct rte_epoll_event elist) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + if (index >= intr_handle->nb_intr) { + RTE_LOG(DEBUG, EAL, "Invalid index %d, max limit %d\n", index, + intr_handle->nb_intr); + rte_errno = ERANGE; + goto fail; + } + + intr_handle->elist[index] = elist; + + return 0; +fail: + return -rte_errno; +} + +int rte_intr_vec_list_alloc(struct rte_intr_handle *intr_handle, + const char *name, int size) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + /* Vector list already allocated */ + if (intr_handle->intr_vec != NULL) + return 0; + + if (size > intr_handle->nb_intr) { + RTE_LOG(DEBUG, EAL, "Invalid size %d, max limit %d\n", size, + intr_handle->nb_intr); + rte_errno = ERANGE; + goto fail; + } + + if (RTE_INTR_INSTANCE_USES_RTE_MEMORY(intr_handle->alloc_flags)) + intr_handle->intr_vec = rte_zmalloc(name, size * sizeof(int), 0); + else + intr_handle->intr_vec = calloc(size, sizeof(int)); + if (intr_handle->intr_vec == NULL) { + RTE_LOG(ERR, EAL, "Failed to allocate %d intr_vec\n", size); + rte_errno = ENOMEM; + goto fail; + } + + intr_handle->vec_list_size = size; + + return 0; +fail: + return -rte_errno; +} + +int rte_intr_vec_list_index_get(const struct rte_intr_handle *intr_handle, + int index) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + if (index >= intr_handle->vec_list_size) { + RTE_LOG(DEBUG, EAL, "Index %d greater than vec list size %d\n", + index, intr_handle->vec_list_size); + rte_errno = ERANGE; + goto fail; + } + + return intr_handle->intr_vec[index]; +fail: + return -rte_errno; +} + +int rte_intr_vec_list_index_set(struct rte_intr_handle *intr_handle, + int index, int vec) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + if (index >= intr_handle->vec_list_size) { + RTE_LOG(DEBUG, EAL, "Index %d greater than vec list size %d\n", + index, intr_handle->vec_list_size); + rte_errno = ERANGE; + goto fail; + } + + intr_handle->intr_vec[index] = vec; + + return 0; +fail: + return -rte_errno; +} + +void rte_intr_vec_list_free(struct rte_intr_handle *intr_handle) +{ + if (intr_handle == NULL) + return; + if (RTE_INTR_INSTANCE_USES_RTE_MEMORY(intr_handle->alloc_flags)) + rte_free(intr_handle->intr_vec); + else + free(intr_handle->intr_vec); + intr_handle->intr_vec = NULL; + intr_handle->vec_list_size = 0; +} + +void *rte_intr_instance_windows_handle_get(struct rte_intr_handle *intr_handle) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + return intr_handle->windows_handle; +fail: + return NULL; +} + +int rte_intr_instance_windows_handle_set(struct rte_intr_handle *intr_handle, + void *windows_handle) +{ + CHECK_VALID_INTR_HANDLE(intr_handle); + + intr_handle->windows_handle = windows_handle; + + return 0; +fail: + return -rte_errno; +} diff --git a/lib/eal/common/meson.build b/lib/eal/common/meson.build index 6d01b0f072..917758cc65 100644 --- a/lib/eal/common/meson.build +++ b/lib/eal/common/meson.build @@ -15,6 +15,7 @@ sources += files( 'eal_common_errno.c', 'eal_common_fbarray.c', 'eal_common_hexdump.c', + 'eal_common_interrupts.c', 'eal_common_launch.c', 'eal_common_lcore.c', 'eal_common_log.c', diff --git a/lib/eal/include/meson.build b/lib/eal/include/meson.build index 88a9eba12f..8e258607b8 100644 --- a/lib/eal/include/meson.build +++ b/lib/eal/include/meson.build @@ -19,6 +19,7 @@ headers += files( 'rte_eal_memconfig.h', 'rte_eal_trace.h', 'rte_errno.h', + 'rte_epoll.h', 'rte_fbarray.h', 'rte_hexdump.h', 'rte_hypervisor.h', diff --git a/lib/eal/include/rte_eal_interrupts.h b/lib/eal/include/rte_eal_interrupts.h index 00bcc19b6d..60bb60ca59 100644 --- a/lib/eal/include/rte_eal_interrupts.h +++ b/lib/eal/include/rte_eal_interrupts.h @@ -39,32 +39,6 @@ enum rte_intr_handle_type { RTE_INTR_HANDLE_MAX /**< count of elements */ }; -#define RTE_INTR_EVENT_ADD 1UL -#define RTE_INTR_EVENT_DEL 2UL - -typedef void (*rte_intr_event_cb_t)(int fd, void *arg); - -struct rte_epoll_data { - uint32_t event; /**< event type */ - void *data; /**< User data */ - rte_intr_event_cb_t cb_fun; /**< IN: callback fun */ - void *cb_arg; /**< IN: callback arg */ -}; - -enum { - RTE_EPOLL_INVALID = 0, - RTE_EPOLL_VALID, - RTE_EPOLL_EXEC, -}; - -/** interrupt epoll event obj, taken by epoll_event.ptr */ -struct rte_epoll_event { - uint32_t status; /**< OUT: event status */ - int fd; /**< OUT: event fd */ - int epfd; /**< OUT: epoll instance the ev associated with */ - struct rte_epoll_data epdata; -}; - /** Handle for interrupts. */ struct rte_intr_handle { RTE_STD_C11 @@ -79,191 +53,20 @@ struct rte_intr_handle { }; int fd; /**< interrupt event file descriptor */ }; - void *handle; /**< device driver handle (Windows) */ + void *windows_handle; /**< device driver handle */ }; + uint32_t alloc_flags; /**< flags passed at allocation */ enum rte_intr_handle_type type; /**< handle type */ uint32_t max_intr; /**< max interrupt requested */ uint32_t nb_efd; /**< number of available efd(event fd) */ uint8_t efd_counter_size; /**< size of efd counter, used for vdev */ + uint16_t nb_intr; + /**< Max vector count, default RTE_MAX_RXTX_INTR_VEC_ID */ int efds[RTE_MAX_RXTX_INTR_VEC_ID]; /**< intr vectors/efds mapping */ struct rte_epoll_event elist[RTE_MAX_RXTX_INTR_VEC_ID]; /**< intr vector epoll event */ + uint16_t vec_list_size; int *intr_vec; /**< intr vector number array */ }; -#define RTE_EPOLL_PER_THREAD -1 /**< to hint using per thread epfd */ - -/** - * It waits for events on the epoll instance. - * Retries if signal received. - * - * @param epfd - * Epoll instance fd on which the caller wait for events. - * @param events - * Memory area contains the events that will be available for the caller. - * @param maxevents - * Up to maxevents are returned, must greater than zero. - * @param timeout - * Specifying a timeout of -1 causes a block indefinitely. - * Specifying a timeout equal to zero cause to return immediately. - * @return - * - On success, returns the number of available event. - * - On failure, a negative value. - */ -int -rte_epoll_wait(int epfd, struct rte_epoll_event *events, - int maxevents, int timeout); - -/** - * It waits for events on the epoll instance. - * Does not retry if signal received. - * - * @param epfd - * Epoll instance fd on which the caller wait for events. - * @param events - * Memory area contains the events that will be available for the caller. - * @param maxevents - * Up to maxevents are returned, must greater than zero. - * @param timeout - * Specifying a timeout of -1 causes a block indefinitely. - * Specifying a timeout equal to zero cause to return immediately. - * @return - * - On success, returns the number of available event. - * - On failure, a negative value. - */ -__rte_experimental -int -rte_epoll_wait_interruptible(int epfd, struct rte_epoll_event *events, - int maxevents, int timeout); - -/** - * It performs control operations on epoll instance referred by the epfd. - * It requests that the operation op be performed for the target fd. - * - * @param epfd - * Epoll instance fd on which the caller perform control operations. - * @param op - * The operation be performed for the target fd. - * @param fd - * The target fd on which the control ops perform. - * @param event - * Describes the object linked to the fd. - * Note: The caller must take care the object deletion after CTL_DEL. - * @return - * - On success, zero. - * - On failure, a negative value. - */ -int -rte_epoll_ctl(int epfd, int op, int fd, - struct rte_epoll_event *event); - -/** - * The function returns the per thread epoll instance. - * - * @return - * epfd the epoll instance referred to. - */ -int -rte_intr_tls_epfd(void); - -/** - * @param intr_handle - * Pointer to the interrupt handle. - * @param epfd - * Epoll instance fd which the intr vector associated to. - * @param op - * The operation be performed for the vector. - * Operation type of {ADD, DEL}. - * @param vec - * RX intr vector number added to the epoll instance wait list. - * @param data - * User raw data. - * @return - * - On success, zero. - * - On failure, a negative value. - */ -int -rte_intr_rx_ctl(struct rte_intr_handle *intr_handle, - int epfd, int op, unsigned int vec, void *data); - -/** - * It deletes registered eventfds. - * - * @param intr_handle - * Pointer to the interrupt handle. - */ -void -rte_intr_free_epoll_fd(struct rte_intr_handle *intr_handle); - -/** - * It enables the packet I/O interrupt event if it's necessary. - * It creates event fd for each interrupt vector when MSIX is used, - * otherwise it multiplexes a single event fd. - * - * @param intr_handle - * Pointer to the interrupt handle. - * @param nb_efd - * Number of interrupt vector trying to enable. - * The value 0 is not allowed. - * @return - * - On success, zero. - * - On failure, a negative value. - */ -int -rte_intr_efd_enable(struct rte_intr_handle *intr_handle, uint32_t nb_efd); - -/** - * It disables the packet I/O interrupt event. - * It deletes registered eventfds and closes the open fds. - * - * @param intr_handle - * Pointer to the interrupt handle. - */ -void -rte_intr_efd_disable(struct rte_intr_handle *intr_handle); - -/** - * The packet I/O interrupt on datapath is enabled or not. - * - * @param intr_handle - * Pointer to the interrupt handle. - */ -int -rte_intr_dp_is_en(struct rte_intr_handle *intr_handle); - -/** - * The interrupt handle instance allows other causes or not. - * Other causes stand for any none packet I/O interrupts. - * - * @param intr_handle - * Pointer to the interrupt handle. - */ -int -rte_intr_allow_others(struct rte_intr_handle *intr_handle); - -/** - * The multiple interrupt vector capability of interrupt handle instance. - * It returns zero if no multiple interrupt vector support. - * - * @param intr_handle - * Pointer to the interrupt handle. - */ -int -rte_intr_cap_multiple(struct rte_intr_handle *intr_handle); - -/** - * @warning - * @b EXPERIMENTAL: this API may change without prior notice - * - * @internal - * Check if currently executing in interrupt context - * - * @return - * - non zero in case of interrupt context - * - zero in case of process context - */ -__rte_experimental -int -rte_thread_is_intr(void); - #endif /* _RTE_EAL_INTERRUPTS_H_ */ diff --git a/lib/eal/include/rte_epoll.h b/lib/eal/include/rte_epoll.h new file mode 100644 index 0000000000..56b7b6bad6 --- /dev/null +++ b/lib/eal/include/rte_epoll.h @@ -0,0 +1,118 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell International Ltd. + */ + +#ifndef __RTE_EPOLL_H__ +#define __RTE_EPOLL_H__ + +/** + * @file + * The rte_epoll provides interfaces functions to add delete events, + * wait poll for an event. + */ + +#include + +#include + +#ifdef __cplusplus +extern "C" { +#endif + +#define RTE_INTR_EVENT_ADD 1UL +#define RTE_INTR_EVENT_DEL 2UL + +typedef void (*rte_intr_event_cb_t)(int fd, void *arg); + +struct rte_epoll_data { + uint32_t event; /**< event type */ + void *data; /**< User data */ + rte_intr_event_cb_t cb_fun; /**< IN: callback fun */ + void *cb_arg; /**< IN: callback arg */ +}; + +enum { + RTE_EPOLL_INVALID = 0, + RTE_EPOLL_VALID, + RTE_EPOLL_EXEC, +}; + +/** interrupt epoll event obj, taken by epoll_event.ptr */ +struct rte_epoll_event { + uint32_t status; /**< OUT: event status */ + int fd; /**< OUT: event fd */ + int epfd; /**< OUT: epoll instance the ev associated with */ + struct rte_epoll_data epdata; +}; + +#define RTE_EPOLL_PER_THREAD -1 /**< to hint using per thread epfd */ + +/** + * It waits for events on the epoll instance. + * Retries if signal received. + * + * @param epfd + * Epoll instance fd on which the caller wait for events. + * @param events + * Memory area contains the events that will be available for the caller. + * @param maxevents + * Up to maxevents are returned, must greater than zero. + * @param timeout + * Specifying a timeout of -1 causes a block indefinitely. + * Specifying a timeout equal to zero cause to return immediately. + * @return + * - On success, returns the number of available event. + * - On failure, a negative value. + */ +int +rte_epoll_wait(int epfd, struct rte_epoll_event *events, + int maxevents, int timeout); + +/** + * It waits for events on the epoll instance. + * Does not retry if signal received. + * + * @param epfd + * Epoll instance fd on which the caller wait for events. + * @param events + * Memory area contains the events that will be available for the caller. + * @param maxevents + * Up to maxevents are returned, must greater than zero. + * @param timeout + * Specifying a timeout of -1 causes a block indefinitely. + * Specifying a timeout equal to zero cause to return immediately. + * @return + * - On success, returns the number of available event. + * - On failure, a negative value. + */ +__rte_experimental +int +rte_epoll_wait_interruptible(int epfd, struct rte_epoll_event *events, + int maxevents, int timeout); + +/** + * It performs control operations on epoll instance referred by the epfd. + * It requests that the operation op be performed for the target fd. + * + * @param epfd + * Epoll instance fd on which the caller perform control operations. + * @param op + * The operation be performed for the target fd. + * @param fd + * The target fd on which the control ops perform. + * @param event + * Describes the object linked to the fd. + * Note: The caller must take care the object deletion after CTL_DEL. + * @return + * - On success, zero. + * - On failure, a negative value. + */ +int +rte_epoll_ctl(int epfd, int op, int fd, + struct rte_epoll_event *event); + +#ifdef __cplusplus +} +#endif + +#endif /* __RTE_EPOLL_H__ */ diff --git a/lib/eal/include/rte_interrupts.h b/lib/eal/include/rte_interrupts.h index cc3bf45d8c..a515a8c073 100644 --- a/lib/eal/include/rte_interrupts.h +++ b/lib/eal/include/rte_interrupts.h @@ -5,8 +5,12 @@ #ifndef _RTE_INTERRUPTS_H_ #define _RTE_INTERRUPTS_H_ +#include + +#include #include #include +#include /** * @file @@ -22,6 +26,15 @@ extern "C" { /** Interrupt handle */ struct rte_intr_handle; +/** Interrupt instance allocation flags + * @see rte_intr_instance_alloc + */ + +/** Interrupt instance will not be shared between primary and secondary processes. */ +#define RTE_INTR_INSTANCE_F_PRIVATE UINT32_C(0) +/** Interrupt instance will be shared between primary and secondary processes. */ +#define RTE_INTR_INSTANCE_F_SHARED RTE_BIT32(0) + /** Function to be registered for the specific interrupt */ typedef void (*rte_intr_callback_fn)(void *cb_arg); @@ -163,6 +176,620 @@ int rte_intr_disable(const struct rte_intr_handle *intr_handle); __rte_experimental int rte_intr_ack(const struct rte_intr_handle *intr_handle); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Check if currently executing in interrupt context + * + * @return + * - non zero in case of interrupt context + * - zero in case of process context + */ +__rte_experimental +int +rte_thread_is_intr(void); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * It allocates memory for interrupt instance. API takes flag as an argument + * which define from where memory should be allocated i.e. using DPDK memory + * management library APIs or normal heap allocation. + * Default memory allocation for event fds and event list array is done which + * can be realloced later based on size of MSIX interrupts supported by a PCI + * device. + * + * This function should be called from application or driver, before calling + * any of the interrupt APIs. + * + * @param flags + * See RTE_INTR_INSTANCE_F_* flags definitions. + * + * @return + * - On success, address of interrupt handle. + * - On failure, NULL. + */ +__rte_experimental +struct rte_intr_handle * +rte_intr_instance_alloc(uint32_t flags); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * This API is used to free the memory allocated for interrupt handle + * resources. + * + * @param intr_handle + * Interrupt handle address. + * + */ +__rte_experimental +void +rte_intr_instance_free(struct rte_intr_handle *intr_handle); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * This API is used to set the fd field of interrupt handle with user provided + * file descriptor. + * + * @param intr_handle + * pointer to the interrupt handle. + * @param fd + * file descriptor value provided by user. + * + * @return + * - On success, zero. + * - On failure, a negative value and rte_errno is set. + */ +__rte_experimental +int +rte_intr_fd_set(struct rte_intr_handle *intr_handle, int fd); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Returns the fd field of the given interrupt handle instance. + * + * @param intr_handle + * pointer to the interrupt handle. + * + * @return + * - On success, fd field. + * - On failure, a negative value. + */ +__rte_experimental +int +rte_intr_fd_get(const struct rte_intr_handle *intr_handle); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * This API is used to set the type field of interrupt handle with user provided + * interrupt type. + * + * @param intr_handle + * pointer to the interrupt handle. + * @param type + * interrupt type + * + * @return + * - On success, zero. + * - On failure, a negative value and rte_errno is set. + */ +__rte_experimental +int +rte_intr_type_set(struct rte_intr_handle *intr_handle, + enum rte_intr_handle_type type); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Returns the type field of the given interrupt handle instance. + * + * @param intr_handle + * pointer to the interrupt handle. + * + * @return + * - On success, interrupt type + * - On failure, RTE_INTR_HANDLE_UNKNOWN. + */ +__rte_experimental +enum rte_intr_handle_type +rte_intr_type_get(const struct rte_intr_handle *intr_handle); + +/** + * @internal + * The function returns the per thread epoll instance. + * + * @return + * epfd the epoll instance referred to. + */ +__rte_internal +int +rte_intr_tls_epfd(void); + +/** + * @internal + * @param intr_handle + * Pointer to the interrupt handle. + * @param epfd + * Epoll instance fd which the intr vector associated to. + * @param op + * The operation be performed for the vector. + * Operation type of {ADD, DEL}. + * @param vec + * RX intr vector number added to the epoll instance wait list. + * @param data + * User raw data. + * @return + * - On success, zero. + * - On failure, a negative value. + */ +__rte_internal +int +rte_intr_rx_ctl(struct rte_intr_handle *intr_handle, + int epfd, int op, unsigned int vec, void *data); + +/** + * @internal + * It deletes registered eventfds. + * + * @param intr_handle + * Pointer to the interrupt handle. + */ +__rte_internal +void +rte_intr_free_epoll_fd(struct rte_intr_handle *intr_handle); + +/** + * @internal + * It enables the packet I/O interrupt event if it's necessary. + * It creates event fd for each interrupt vector when MSIX is used, + * otherwise it multiplexes a single event fd. + * + * @param intr_handle + * Pointer to the interrupt handle. + * @param nb_efd + * Number of interrupt vector trying to enable. + * The value 0 is not allowed. + * @return + * - On success, zero. + * - On failure, a negative value. + */ +__rte_internal +int +rte_intr_efd_enable(struct rte_intr_handle *intr_handle, uint32_t nb_efd); + +/** + * @internal + * It disables the packet I/O interrupt event. + * It deletes registered eventfds and closes the open fds. + * + * @param intr_handle + * Pointer to the interrupt handle. + */ +__rte_internal +void +rte_intr_efd_disable(struct rte_intr_handle *intr_handle); + +/** + * @internal + * The packet I/O interrupt on datapath is enabled or not. + * + * @param intr_handle + * Pointer to the interrupt handle. + */ +__rte_internal +int +rte_intr_dp_is_en(struct rte_intr_handle *intr_handle); + +/** + * @internal + * The interrupt handle instance allows other causes or not. + * Other causes stand for any none packet I/O interrupts. + * + * @param intr_handle + * Pointer to the interrupt handle. + */ +__rte_internal +int +rte_intr_allow_others(struct rte_intr_handle *intr_handle); + +/** + * @internal + * The multiple interrupt vector capability of interrupt handle instance. + * It returns zero if no multiple interrupt vector support. + * + * @param intr_handle + * Pointer to the interrupt handle. + */ +__rte_internal +int +rte_intr_cap_multiple(struct rte_intr_handle *intr_handle); + +/** + * @internal + * Creates a clone of src by allocating a new handle and copying src content. + * + * @param src + * Source interrupt handle to be cloned. + * + * @return + * - On success, address of interrupt handle. + * - On failure, NULL. + */ +__rte_internal +struct rte_intr_handle * +rte_intr_instance_dup(const struct rte_intr_handle *src); + +/** + * @internal + * This API is used to set the device fd field of interrupt handle with user + * provided dev fd. Device fd corresponds to VFIO device fd or UIO config fd. + * + * @param intr_handle + * pointer to the interrupt handle. + * @param fd + * interrupt type + * + * @return + * - On success, zero. + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +int +rte_intr_dev_fd_set(struct rte_intr_handle *intr_handle, int fd); + +/** + * @internal + * Returns the device fd field of the given interrupt handle instance. + * + * @param intr_handle + * pointer to the interrupt handle. + * + * @return + * - On success, dev fd. + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +int +rte_intr_dev_fd_get(const struct rte_intr_handle *intr_handle); + +/** + * @internal + * This API is used to set the max intr field of interrupt handle with user + * provided max intr value. + * + * @param intr_handle + * pointer to the interrupt handle. + * @param max_intr + * interrupt type + * + * @return + * - On success, zero. + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +int +rte_intr_max_intr_set(struct rte_intr_handle *intr_handle, int max_intr); + +/** + * @internal + * Returns the max intr field of the given interrupt handle instance. + * + * @param intr_handle + * pointer to the interrupt handle. + * + * @return + * - On success, max intr. + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +int +rte_intr_max_intr_get(const struct rte_intr_handle *intr_handle); + +/** + * @internal + * This API is used to set the number of event fd field of interrupt handle + * with user provided available event file descriptor value. + * + * @param intr_handle + * pointer to the interrupt handle. + * @param nb_efd + * Available event fd + * + * @return + * - On success, zero. + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +int +rte_intr_nb_efd_set(struct rte_intr_handle *intr_handle, int nb_efd); + +/** + * @internal + * Returns the number of available event fd field of the given interrupt handle + * instance. + * + * @param intr_handle + * pointer to the interrupt handle. + * + * @return + * - On success, nb_efd + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +int +rte_intr_nb_efd_get(const struct rte_intr_handle *intr_handle); + +/** + * @internal + * Returns the number of interrupt vector field of the given interrupt handle + * instance. This field is to configured on device probe time, and based on + * this value efds and elist arrays are dynamically allocated. By default + * this value is set to RTE_MAX_RXTX_INTR_VEC_ID. + * For eg. in case of PCI device, its msix size is queried and efds/elist + * arrays are allocated accordingly. + * + * @param intr_handle + * pointer to the interrupt handle. + * + * @return + * - On success, nb_intr + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +int +rte_intr_nb_intr_get(const struct rte_intr_handle *intr_handle); + +/** + * @internal + * This API is used to set the event fd counter size field of interrupt handle + * with user provided efd counter size. + * + * @param intr_handle + * pointer to the interrupt handle. + * @param efd_counter_size + * size of efd counter. + * + * @return + * - On success, zero. + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +int +rte_intr_efd_counter_size_set(struct rte_intr_handle *intr_handle, + uint8_t efd_counter_size); + +/** + * @internal + * Returns the event fd counter size field of the given interrupt handle + * instance. + * + * @param intr_handle + * pointer to the interrupt handle. + * + * @return + * - On success, efd_counter_size + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +int +rte_intr_efd_counter_size_get(const struct rte_intr_handle *intr_handle); + +/** + * @internal + * This API is used to set the event fd array index with the given fd. + * + * @param intr_handle + * pointer to the interrupt handle. + * @param index + * efds array index to be set + * @param fd + * event fd + * + * @return + * - On success, zero. + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +int +rte_intr_efds_index_set(struct rte_intr_handle *intr_handle, int index, int fd); + +/** + * @internal + * Returns the fd value of event fds array at a given index. + * + * @param intr_handle + * pointer to the interrupt handle. + * @param index + * efds array index to be returned + * + * @return + * - On success, fd + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +int +rte_intr_efds_index_get(const struct rte_intr_handle *intr_handle, int index); + +/** + * @internal + * This API is used to set the epoll event object array index with the given + * elist instance. + * + * @param intr_handle + * pointer to the interrupt handle. + * @param index + * elist array index to be set + * @param elist + * epoll event instance of struct rte_epoll_event + * + * @return + * - On success, zero. + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +int +rte_intr_elist_index_set(struct rte_intr_handle *intr_handle, int index, + struct rte_epoll_event elist); + +/** + * @internal + * Returns the address of epoll event instance from elist array at a given + * index. + * + * @param intr_handle + * pointer to the interrupt handle. + * @param index + * elist array index to be returned + * + * @return + * - On success, elist + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +struct rte_epoll_event * +rte_intr_elist_index_get(struct rte_intr_handle *intr_handle, int index); + +/** + * @internal + * Allocates the memory of interrupt vector list array, with size defining the + * number of elements required in the array. + * + * @param intr_handle + * pointer to the interrupt handle. + * @param name + * Name assigned to the allocation, or NULL. + * @param size + * Number of element required in the array. + * + * @return + * - On success, zero + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +int +rte_intr_vec_list_alloc(struct rte_intr_handle *intr_handle, const char *name, + int size); + +/** + * @internal + * Sets the vector value at given index of interrupt vector list field of given + * interrupt handle. + * + * @param intr_handle + * pointer to the interrupt handle. + * @param index + * intr_vec array index to be set + * @param vec + * Interrupt vector value. + * + * @return + * - On success, zero + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +int +rte_intr_vec_list_index_set(struct rte_intr_handle *intr_handle, int index, + int vec); + +/** + * @internal + * Returns the vector value at the given index of interrupt vector list array. + * + * @param intr_handle + * pointer to the interrupt handle. + * @param index + * intr_vec array index to be returned + * + * @return + * - On success, interrupt vector + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +int +rte_intr_vec_list_index_get(const struct rte_intr_handle *intr_handle, + int index); + +/** + * @internal + * Frees the memory allocated for interrupt vector list array. + * + * @param intr_handle + * pointer to the interrupt handle. + * + * @return + * - On success, zero + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +void +rte_intr_vec_list_free(struct rte_intr_handle *intr_handle); + +/** + * @internal + * Reallocates the size efds and elist array based on size provided by user. + * By default efds and elist array are allocated with default size + * RTE_MAX_RXTX_INTR_VEC_ID on interrupt handle array creation. Later on device + * probe, device may have capability of more interrupts than + * RTE_MAX_RXTX_INTR_VEC_ID. Using this API, PMDs can reallocate the arrays as + * per the max interrupts capability of device. + * + * @param intr_handle + * pointer to the interrupt handle. + * @param size + * efds and elist array size. + * + * @return + * - On success, zero + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +int +rte_intr_event_list_update(struct rte_intr_handle *intr_handle, int size); + +/** + * @internal + * This API returns the Windows handle of the given interrupt instance. + * + * @param intr_handle + * pointer to the interrupt handle. + * + * @return + * - On success, Windows handle. + * - On failure, NULL. + */ +__rte_internal +void * +rte_intr_instance_windows_handle_get(struct rte_intr_handle *intr_handle); + +/** + * @internal + * This API set the Windows handle for the given interrupt instance. + * + * @param intr_handle + * pointer to the interrupt handle. + * @param windows_handle + * Windows handle to be set. + * + * @return + * - On success, zero + * - On failure, a negative value and rte_errno is set. + */ +__rte_internal +int +rte_intr_instance_windows_handle_set(struct rte_intr_handle *intr_handle, + void *windows_handle); + #ifdef __cplusplus } #endif diff --git a/lib/eal/version.map b/lib/eal/version.map index 38f7de83e1..9d43655b66 100644 --- a/lib/eal/version.map +++ b/lib/eal/version.map @@ -109,18 +109,10 @@ DPDK_22 { rte_hexdump; rte_hypervisor_get; rte_hypervisor_get_name; # WINDOWS_NO_EXPORT - rte_intr_allow_others; rte_intr_callback_register; rte_intr_callback_unregister; - rte_intr_cap_multiple; rte_intr_disable; - rte_intr_dp_is_en; - rte_intr_efd_disable; - rte_intr_efd_enable; rte_intr_enable; - rte_intr_free_epoll_fd; - rte_intr_rx_ctl; - rte_intr_tls_epfd; rte_keepalive_create; # WINDOWS_NO_EXPORT rte_keepalive_dispatch_pings; # WINDOWS_NO_EXPORT rte_keepalive_mark_alive; # WINDOWS_NO_EXPORT @@ -420,12 +412,49 @@ EXPERIMENTAL { # added in 21.08 rte_power_monitor_multi; # WINDOWS_NO_EXPORT + + # added in 21.11 + rte_intr_fd_get; + rte_intr_fd_set; + rte_intr_instance_alloc; + rte_intr_instance_free; + rte_intr_type_get; + rte_intr_type_set; }; INTERNAL { global: rte_firmware_read; + rte_intr_allow_others; + rte_intr_cap_multiple; + rte_intr_dev_fd_get; + rte_intr_dev_fd_set; + rte_intr_dp_is_en; + rte_intr_efd_counter_size_set; + rte_intr_efd_counter_size_get; + rte_intr_efd_disable; + rte_intr_efd_enable; + rte_intr_efds_index_get; + rte_intr_efds_index_set; + rte_intr_elist_index_get; + rte_intr_elist_index_set; + rte_intr_event_list_update; + rte_intr_free_epoll_fd; + rte_intr_instance_dup; + rte_intr_instance_windows_handle_get; + rte_intr_instance_windows_handle_set; + rte_intr_max_intr_get; + rte_intr_max_intr_set; + rte_intr_nb_efd_get; + rte_intr_nb_efd_set; + rte_intr_nb_intr_get; + rte_intr_rx_ctl; + rte_intr_tls_epfd; + rte_intr_vec_list_alloc; + rte_intr_vec_list_free; + rte_intr_vec_list_index_get; + rte_intr_vec_list_index_set; rte_mem_lock; rte_mem_map; rte_mem_page_size; -- 2.23.0