From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CE2E4A0561; Thu, 18 Mar 2021 02:01:37 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 87A1340698; Thu, 18 Mar 2021 02:01:37 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id 333A040683 for ; Thu, 18 Mar 2021 02:01:36 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1059) id 7A7B7209C371; Wed, 17 Mar 2021 18:01:35 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 7A7B7209C371 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1616029295; bh=kSCMbfGLflvCagsAzyaYiUrrhc4j83FjM1oj75aYT0M=; h=From:To:Cc:Subject:Date:From; b=b5xfUviiIu3g6YCylonfeRrpokUxAKhBu7paEntl6D60wIfl4miCmQ7r+DZ0SILeM m4KFsUErpSNv//p+fYGExQ9qeCb+685LJKvTijM8IlJuxatqaC+kQkcGSTLliyTX2s kTBvC1TZW8YIpwgmwRsyxlok1bqb9UQAkeBUkEqc= From: Narcisa Ana Maria Vasile To: dev@dpdk.org, thomas@monjalon.net, dmitry.kozliuk@gmail.com, khot@microsoft.com, navasile@microsoft.com, dmitrym@microsoft.com, roretzla@microsoft.com, ocardona@microsoft.com Cc: bruce.richardson@intel.com, david.marchand@redhat.com, pallavi.kadam@intel.com Date: Wed, 17 Mar 2021 18:00:37 -0700 Message-Id: <1616029240-26588-1-git-send-email-navasile@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH 0/3] eal: Add new API for threading X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Narcisa Vasile EAL thread API **Problem Statement** DPDK currently uses the pthread interface to create and manage threads. Windows does not support the POSIX thread programming model, so it currently relies on a header file that hides the Windows calls under pthread API. Given that EAL should isolate the environment specifics from the applications and libraries and intermediate the communication with the operating systems, a new EAL interface is needed for thread management. **Goals** * Introduce a generic EAL API for threading support that will remove the current Windows pthread.h shim. * Replace references to pthread_* across the DPDK codebase with the new rte_thread_* API. * Allow users to choose between using the rte_thread_* API or a 3rd party thread library through a configuration option. **Design plan** New API main files: * rte_thread.h (librte_eal/include) * rte_thread_types.h (librte_eal/include) * rte_thread_windows_types.h (librte_eal/windows/include) * rte_thread.c (librte_eal/windows) * rte_thread.c (librte_eal/common) For flexibility, the user is offered the option of either using the rte_thread_* API or a 3rd party thread library, through a meson flag “use_external_thread_lib”. By default, this flag is set to FALSE, which means Windows libraries and applications will use the rte_thread_* API for managing threads. If compiling on Windows and the new EAL API will be used, the following files will be parsed: * include/rte_thread.h * windows/include/rte_thread_windows_types.h * windows/rte_thread.c In all other cases, the compilation/parsing includes the following files: * include/rte_thread.h * include/rte_thread_types.h * common/rte_thread.c **A schematic example of the design** -------------------------------------------------- lib/librte_eal/include/rte_thread.h int rte_thread_create(); lib/librte_eal/common/rte_thread.c int rte_thread_create() { return pthread_create(); } lib/librte_eal/windows/rte_thread.c int rte_thread_create() { return CreateThread(); } lib/librte_eal/windows/meson.build if get_option('use_external_thread_lib') sources += 'librte_eal/common/rte_thread.c' else sources += 'librte_eal/windows/rte_thread.c' endif ----------------------------------------------------- **Thread attributes** When or after a thread is created, specific characteristics of the thread can be adjusted. Given that the thread characteristics that are of interest for DPDK applications are affinity and priority, the following structure that represents thread attributes has been defined: typedef struct { enum rte_thread_priority priority; rte_cpuset_t cpuset; } rte_thread_attr_t; The *rte_thread_create()* function can optionally receive an rte_thread_attr_t object that will cause the thread to be created with the affinity and priority described by the attributes object. If no rte_thread_attr_t is passed (parameter is NULL), the default affinity and priority are used. An rte_thread_attr_t object can also be set to the default values by calling *rte_thread_attr_init()*. *Priority* is represented through an enum that currently advertises two values for priority: - RTE_THREAD_PRIORITY_NORMAL - RTE_THREAD_PRIORITY_REALTIME_CRITICAL The enum can be extended to allow for multiple priority levels. rte_thread_set_priority - sets the priority of a thread rte_thread_attr_set_priority - updates an rte_thread_attr_t object with a new value for priority The user can choose thread priority through an EAL parameter, when starting an application. The two options above are available: --thread-prio normal --thread-prio realtime Example: ./dpdk-l2fwd -l 0-3 -n 4 –thread-prio normal -- -q 8 -p ffff *Affinity* is described by the already known “rte_cpuset_t” type. rte_thread_attr_set/get_affinity - sets/gets the affinity field in a rte_thread_attr_t object rte_thread_set/get_affinity – sets/gets the affinity of a thread **Errors** A translation function that maps Windows error codes to errno-style error codes is provided. **Future work** Note that this patchset was focused on introducing new API that will remove the Windows pthread.h shim. In DPDK, there are still a few references to pthread_* that were not implemented in the shim. The long term plan is for EAL to provide full threading support: * Adding support for conditional variables * Additional functionality offered by pthread_* (such as pthread_setname_np, etc.) * Static mutex initializers are not used on Windows. If we must continue using them, they need to be platform dependent and an implementation will need to be provided for Windows. Narcisa Vasile (3): Add EAL threads API Add EAL argument for setting thread priority Enable the new EAL thread API and rename app/test/process.h | 6 +- app/test/test_lcores.c | 16 +- app/test/test_link_bonding.c | 10 +- app/test/test_lpm_perf.c | 12 +- config/meson.build | 4 + drivers/bus/dpaa/base/qbman/bman_driver.c | 4 +- drivers/bus/dpaa/base/qbman/dpaa_sys.c | 14 +- drivers/bus/dpaa/base/qbman/process.c | 6 +- drivers/bus/fslmc/portal/dpaa2_hw_dpio.c | 4 +- drivers/compress/mlx5/mlx5_compress.c | 10 +- drivers/event/dlb/pf/base/dlb_osdep.h | 2 +- drivers/event/dlb2/pf/base/dlb2_osdep.h | 2 +- drivers/net/af_xdp/rte_eth_af_xdp.c | 18 +- drivers/net/ark/ark_ethdev.c | 4 +- drivers/net/atlantic/atl_ethdev.c | 4 +- drivers/net/atlantic/atl_types.h | 5 +- .../net/atlantic/hw_atl/hw_atl_utils_fw2x.c | 24 +- drivers/net/axgbe/axgbe_dev.c | 8 +- drivers/net/axgbe/axgbe_ethdev.c | 8 +- drivers/net/axgbe/axgbe_ethdev.h | 8 +- drivers/net/axgbe/axgbe_i2c.c | 4 +- drivers/net/axgbe/axgbe_mdio.c | 8 +- drivers/net/axgbe/axgbe_phy_impl.c | 6 +- drivers/net/bnxt/bnxt.h | 16 +- drivers/net/bnxt/bnxt_cpr.c | 4 +- drivers/net/bnxt/bnxt_ethdev.c | 54 +- drivers/net/bnxt/bnxt_irq.c | 8 +- drivers/net/bnxt/bnxt_reps.c | 10 +- drivers/net/bnxt/tf_ulp/bnxt_ulp.c | 34 +- drivers/net/bnxt/tf_ulp/bnxt_ulp.h | 4 +- drivers/net/bnxt/tf_ulp/ulp_fc_mgr.c | 24 +- drivers/net/bnxt/tf_ulp/ulp_fc_mgr.h | 2 +- drivers/net/ena/base/ena_plat_dpdk.h | 8 +- drivers/net/enic/enic.h | 2 +- drivers/net/hns3/hns3_ethdev.h | 2 +- drivers/net/hns3/hns3_ethdev_vf.c | 2 +- drivers/net/hns3/hns3_mbx.c | 2 +- drivers/net/ice/ice_dcf_parent.c | 4 +- drivers/net/ipn3ke/ipn3ke_representor.c | 6 +- drivers/net/ixgbe/ixgbe_ethdev.c | 2 +- drivers/net/ixgbe/ixgbe_ethdev.h | 2 +- drivers/net/kni/rte_eth_kni.c | 6 +- drivers/net/mlx5/linux/mlx5_os.c | 2 +- drivers/net/mlx5/mlx5.c | 20 +- drivers/net/mlx5/mlx5.h | 2 +- drivers/net/mlx5/mlx5_txpp.c | 8 +- drivers/net/mlx5/windows/mlx5_flow_os.c | 10 +- drivers/net/mlx5/windows/mlx5_os.c | 2 +- drivers/net/qede/base/bcm_osal.h | 8 +- drivers/net/vhost/rte_eth_vhost.c | 22 +- .../net/virtio/virtio_user/virtio_user_dev.c | 30 +- .../net/virtio/virtio_user/virtio_user_dev.h | 2 +- drivers/raw/ifpga/ifpga_rawdev.c | 8 +- drivers/vdpa/ifc/ifcvf_vdpa.c | 36 +- drivers/vdpa/mlx5/mlx5_vdpa.c | 24 +- drivers/vdpa/mlx5/mlx5_vdpa.h | 6 +- drivers/vdpa/mlx5/mlx5_vdpa_event.c | 54 +- examples/kni/main.c | 4 +- examples/vhost/main.c | 2 +- examples/vhost_blk/vhost_blk.c | 10 +- lib/librte_eal/common/eal_common_options.c | 32 +- lib/librte_eal/common/eal_common_proc.c | 48 +- lib/librte_eal/common/eal_common_thread.c | 47 +- lib/librte_eal/common/eal_common_trace.c | 2 +- lib/librte_eal/common/eal_internal_cfg.h | 2 + lib/librte_eal/common/eal_options.h | 2 + lib/librte_eal/common/eal_private.h | 2 +- lib/librte_eal/common/malloc_mp.c | 32 +- lib/librte_eal/common/meson.build | 1 + lib/librte_eal/common/rte_thread.c | 346 ++++++++++++ lib/librte_eal/freebsd/eal.c | 33 +- lib/librte_eal/freebsd/eal_alarm.c | 12 +- lib/librte_eal/freebsd/eal_interrupts.c | 4 +- lib/librte_eal/freebsd/eal_thread.c | 8 +- lib/librte_eal/include/rte_lcore.h | 8 +- lib/librte_eal/include/rte_per_lcore.h | 2 - lib/librte_eal/include/rte_thread.h | 323 ++++++++++- lib/librte_eal/include/rte_thread_types.h | 20 + lib/librte_eal/linux/eal.c | 36 +- lib/librte_eal/linux/eal_alarm.c | 8 +- lib/librte_eal/linux/eal_interrupts.c | 4 +- lib/librte_eal/linux/eal_thread.c | 8 +- lib/librte_eal/linux/eal_timer.c | 2 +- lib/librte_eal/rte_eal_exports.def | 19 + lib/librte_eal/unix/meson.build | 1 - lib/librte_eal/unix/rte_thread.c | 86 --- lib/librte_eal/version.map | 21 + lib/librte_eal/windows/eal.c | 24 +- lib/librte_eal/windows/eal_interrupts.c | 6 +- lib/librte_eal/windows/eal_lcore.c | 167 ++++-- lib/librte_eal/windows/eal_thread.c | 24 +- lib/librte_eal/windows/eal_windows.h | 20 +- lib/librte_eal/windows/include/meson.build | 1 + lib/librte_eal/windows/include/pthread.h | 186 ------- .../include/rte_windows_thread_types.h | 19 + lib/librte_eal/windows/include/sched.h | 2 +- lib/librte_eal/windows/meson.build | 7 +- lib/librte_eal/windows/rte_thread.c | 507 +++++++++++++++++- lib/librte_ethdev/rte_ethdev.c | 4 +- lib/librte_ethdev/rte_ethdev_core.h | 3 +- lib/librte_ethdev/rte_flow.c | 4 +- .../rte_event_eth_rx_adapter.c | 6 +- lib/librte_vhost/fd_man.c | 40 +- lib/librte_vhost/fd_man.h | 6 +- lib/librte_vhost/socket.c | 124 ++--- lib/librte_vhost/vhost.c | 10 +- meson_options.txt | 2 + 107 files changed, 2001 insertions(+), 899 deletions(-) create mode 100644 lib/librte_eal/common/rte_thread.c create mode 100644 lib/librte_eal/include/rte_thread_types.h delete mode 100644 lib/librte_eal/unix/rte_thread.c delete mode 100644 lib/librte_eal/windows/include/pthread.h create mode 100644 lib/librte_eal/windows/include/rte_windows_thread_types.h -- 2.30.0.vfs.0.2