From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id CAE2A5A64 for ; Tue, 14 Jul 2015 10:57:18 +0200 (CEST) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga101.jf.intel.com with ESMTP; 14 Jul 2015 01:57:17 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.15,470,1432623600"; d="scan'208";a="764039325" Received: from irvmail001.ir.intel.com ([163.33.26.43]) by orsmga002.jf.intel.com with ESMTP; 14 Jul 2015 01:57:14 -0700 Received: from sivswdev02.ir.intel.com (sivswdev02.ir.intel.com [10.237.217.46]) by irvmail001.ir.intel.com (8.14.3/8.13.6/MailSET/Hub) with ESMTP id t6E8vDnE030984 for ; Tue, 14 Jul 2015 09:57:13 +0100 Received: from sivswdev02.ir.intel.com (localhost [127.0.0.1]) by sivswdev02.ir.intel.com with ESMTP id t6E8vDlF024791 for ; Tue, 14 Jul 2015 09:57:13 +0100 Received: (from smonroy@localhost) by sivswdev02.ir.intel.com with id t6E8vDPa024787 for dev@dpdk.org; Tue, 14 Jul 2015 09:57:13 +0100 From: Sergio Gonzalez Monroy To: dev@dpdk.org Date: Tue, 14 Jul 2015 09:57:05 +0100 Message-Id: <1436864233-24746-2-git-send-email-sergio.gonzalez.monroy@intel.com> X-Mailer: git-send-email 1.8.5.4 In-Reply-To: <1436864233-24746-1-git-send-email-sergio.gonzalez.monroy@intel.com> References: <1435917343-23032-1-git-send-email-sergio.gonzalez.monroy@intel.com> <1436864233-24746-1-git-send-email-sergio.gonzalez.monroy@intel.com> Subject: [dpdk-dev] [PATCH v8 1/9] eal: move librte_malloc to eal/common X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Jul 2015 08:57:20 -0000 Move malloc inside eal and create a new section in MAINTAINERS file for Memory Allocation in EAL. Create a dummy malloc library to avoid breaking applications that have librte_malloc in their DT_NEEDED entries. This is the first step towards using malloc to allocate memory directly from memsegs. Thus, memzones would allocate memory through malloc, allowing to free memzones. Signed-off-by: Sergio Gonzalez Monroy --- MAINTAINERS | 22 +- config/common_bsdapp | 9 +- config/common_linuxapp | 9 +- drivers/net/af_packet/Makefile | 1 - drivers/net/bonding/Makefile | 1 - drivers/net/e1000/Makefile | 2 +- drivers/net/enic/Makefile | 2 +- drivers/net/fm10k/Makefile | 2 +- drivers/net/i40e/Makefile | 2 +- drivers/net/ixgbe/Makefile | 2 +- drivers/net/mlx4/Makefile | 1 - drivers/net/null/Makefile | 1 - drivers/net/pcap/Makefile | 1 - drivers/net/virtio/Makefile | 2 +- drivers/net/vmxnet3/Makefile | 2 +- drivers/net/xenvirt/Makefile | 2 +- lib/Makefile | 2 +- lib/librte_acl/Makefile | 2 +- lib/librte_eal/bsdapp/eal/Makefile | 4 +- lib/librte_eal/bsdapp/eal/rte_eal_version.map | 13 + lib/librte_eal/common/Makefile | 1 + lib/librte_eal/common/include/rte_malloc.h | 342 ++++++++++++++++++++++++ lib/librte_eal/common/malloc_elem.c | 320 ++++++++++++++++++++++ lib/librte_eal/common/malloc_elem.h | 190 +++++++++++++ lib/librte_eal/common/malloc_heap.c | 208 ++++++++++++++ lib/librte_eal/common/malloc_heap.h | 70 +++++ lib/librte_eal/common/rte_malloc.c | 260 ++++++++++++++++++ lib/librte_eal/linuxapp/eal/Makefile | 4 +- lib/librte_eal/linuxapp/eal/rte_eal_version.map | 13 + lib/librte_hash/Makefile | 2 +- lib/librte_lpm/Makefile | 2 +- lib/librte_malloc/Makefile | 6 +- lib/librte_malloc/malloc_elem.c | 320 ---------------------- lib/librte_malloc/malloc_elem.h | 190 ------------- lib/librte_malloc/malloc_heap.c | 208 -------------- lib/librte_malloc/malloc_heap.h | 70 ----- lib/librte_malloc/rte_malloc.c | 228 +--------------- lib/librte_malloc/rte_malloc.h | 342 ------------------------ lib/librte_malloc/rte_malloc_version.map | 16 -- lib/librte_mempool/Makefile | 2 - lib/librte_port/Makefile | 1 - lib/librte_ring/Makefile | 3 +- lib/librte_table/Makefile | 1 - 43 files changed, 1455 insertions(+), 1426 deletions(-) create mode 100644 lib/librte_eal/common/include/rte_malloc.h create mode 100644 lib/librte_eal/common/malloc_elem.c create mode 100644 lib/librte_eal/common/malloc_elem.h create mode 100644 lib/librte_eal/common/malloc_heap.c create mode 100644 lib/librte_eal/common/malloc_heap.h create mode 100644 lib/librte_eal/common/rte_malloc.c delete mode 100644 lib/librte_malloc/malloc_elem.c delete mode 100644 lib/librte_malloc/malloc_elem.h delete mode 100644 lib/librte_malloc/malloc_heap.c delete mode 100644 lib/librte_malloc/malloc_heap.h delete mode 100644 lib/librte_malloc/rte_malloc.h diff --git a/MAINTAINERS b/MAINTAINERS index a15105d..52892d0 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -83,12 +83,9 @@ F: app/test/test_debug.c F: app/test/test_devargs.c F: app/test/test_eal* F: app/test/test_errno.c -F: app/test/test_func_reentrancy.c F: app/test/test_interrupts.c F: app/test/test_logs.c F: app/test/test_memcpy* -F: app/test/test_memory.c -F: app/test/test_memzone.c F: app/test/test_pci.c F: app/test/test_per_lcore.c F: app/test/test_prefetch.c @@ -98,6 +95,19 @@ F: app/test/test_string_fns.c F: app/test/test_tailq.c F: app/test/test_version.c +Memory Allocation +M: Sergio Gonzalez Monroy +F: lib/librte_eal/common/include/rte_mem* +F: lib/librte_eal/common/include/rte_malloc.h +F: lib/librte_eal/common/*malloc* +F: lib/librte_eal/common/eal_common_mem* +F: lib/librte_eal/common/eal_hugepages.h +F: doc/guides/prog_guide/malloc_lib.rst +F: app/test/test_func_reentrancy.c +F: app/test/test_malloc.c +F: app/test/test_memory.c +F: app/test/test_memzone.c + Secondary process K: RTE_PROC_ F: doc/guides/prog_guide/multi_proc_support.rst @@ -161,12 +171,6 @@ F: lib/librte_eal/bsdapp/nic_uio/ Core Libraries -------------- -Dynamic memory -F: lib/librte_malloc/ -F: doc/guides/prog_guide/malloc_lib.rst -F: app/test/test_malloc.c -F: app/test/test_func_reentrancy.c - Memory pool M: Olivier Matz F: lib/librte_mempool/ diff --git a/config/common_bsdapp b/config/common_bsdapp index 7112f1c..5bb7f55 100644 --- a/config/common_bsdapp +++ b/config/common_bsdapp @@ -106,6 +106,8 @@ CONFIG_RTE_LOG_LEVEL=8 CONFIG_RTE_LOG_HISTORY=256 CONFIG_RTE_EAL_ALLOW_INV_SOCKET_ID=n CONFIG_RTE_EAL_ALWAYS_PANIC_ON_ERROR=n +CONFIG_RTE_MALLOC_DEBUG=n +CONFIG_RTE_MALLOC_MEMZONE_SIZE=11M # # FreeBSD contiguous memory driver settings @@ -308,13 +310,6 @@ CONFIG_RTE_LIBRTE_TIMER=y CONFIG_RTE_LIBRTE_TIMER_DEBUG=n # -# Compile librte_malloc -# -CONFIG_RTE_LIBRTE_MALLOC=y -CONFIG_RTE_LIBRTE_MALLOC_DEBUG=n -CONFIG_RTE_MALLOC_MEMZONE_SIZE=11M - -# # Compile librte_cfgfile # CONFIG_RTE_LIBRTE_CFGFILE=y diff --git a/config/common_linuxapp b/config/common_linuxapp index 4d90d35..7b57044 100644 --- a/config/common_linuxapp +++ b/config/common_linuxapp @@ -109,6 +109,8 @@ CONFIG_RTE_EAL_ALLOW_INV_SOCKET_ID=n CONFIG_RTE_EAL_ALWAYS_PANIC_ON_ERROR=n CONFIG_RTE_EAL_IGB_UIO=y CONFIG_RTE_EAL_VFIO=y +CONFIG_RTE_MALLOC_DEBUG=n +CONFIG_RTE_MALLOC_MEMZONE_SIZE=11M # # Special configurations in PCI Config Space for high performance @@ -316,13 +318,6 @@ CONFIG_RTE_LIBRTE_TIMER=y CONFIG_RTE_LIBRTE_TIMER_DEBUG=n # -# Compile librte_malloc -# -CONFIG_RTE_LIBRTE_MALLOC=y -CONFIG_RTE_LIBRTE_MALLOC_DEBUG=n -CONFIG_RTE_MALLOC_MEMZONE_SIZE=11M - -# # Compile librte_cfgfile # CONFIG_RTE_LIBRTE_CFGFILE=y diff --git a/drivers/net/af_packet/Makefile b/drivers/net/af_packet/Makefile index f0bf537..ce5d239 100644 --- a/drivers/net/af_packet/Makefile +++ b/drivers/net/af_packet/Makefile @@ -58,7 +58,6 @@ SYMLINK-y-include += rte_eth_af_packet.h # this lib depends upon: DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_mbuf DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_ether -DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_malloc DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += lib/librte_kvargs include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/net/bonding/Makefile b/drivers/net/bonding/Makefile index 83ccce3..dee0875 100644 --- a/drivers/net/bonding/Makefile +++ b/drivers/net/bonding/Makefile @@ -61,7 +61,6 @@ SYMLINK-y-include += rte_eth_bond_8023ad.h # this lib depends upon: DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += lib/librte_mbuf DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += lib/librte_ether -DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += lib/librte_malloc DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += lib/librte_eal DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += lib/librte_kvargs diff --git a/drivers/net/e1000/Makefile b/drivers/net/e1000/Makefile index 3d525fa..ccd2b7b 100644 --- a/drivers/net/e1000/Makefile +++ b/drivers/net/e1000/Makefile @@ -94,6 +94,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_EM_PMD) += em_rxtx.c # this lib depends upon: DEPDIRS-$(CONFIG_RTE_LIBRTE_E1000_PMD) += lib/librte_eal lib/librte_ether DEPDIRS-$(CONFIG_RTE_LIBRTE_E1000_PMD) += lib/librte_mempool lib/librte_mbuf -DEPDIRS-$(CONFIG_RTE_LIBRTE_E1000_PMD) += lib/librte_net lib/librte_malloc +DEPDIRS-$(CONFIG_RTE_LIBRTE_E1000_PMD) += lib/librte_net include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/net/enic/Makefile b/drivers/net/enic/Makefile index 52334c9..f0ee093 100644 --- a/drivers/net/enic/Makefile +++ b/drivers/net/enic/Makefile @@ -65,7 +65,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_ENIC_PMD) += base/vnic_rss.c # this lib depends upon: DEPDIRS-$(CONFIG_RTE_LIBRTE_ENIC_PMD) += lib/librte_eal lib/librte_ether DEPDIRS-$(CONFIG_RTE_LIBRTE_ENIC_PMD) += lib/librte_mempool lib/librte_mbuf -DEPDIRS-$(CONFIG_RTE_LIBRTE_ENIC_PMD) += lib/librte_net lib/librte_malloc +DEPDIRS-$(CONFIG_RTE_LIBRTE_ENIC_PMD) += lib/librte_net DEPDIRS-$(CONFIG_RTE_LIBRTE_ENIC_PMD) += lib/librte_hash include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/net/fm10k/Makefile b/drivers/net/fm10k/Makefile index 7395933..a4a8f56 100644 --- a/drivers/net/fm10k/Makefile +++ b/drivers/net/fm10k/Makefile @@ -97,6 +97,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += fm10k_api.c # this lib depends upon: DEPDIRS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += lib/librte_eal lib/librte_ether DEPDIRS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += lib/librte_mempool lib/librte_mbuf -DEPDIRS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += lib/librte_net lib/librte_malloc +DEPDIRS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += lib/librte_net include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/net/i40e/Makefile b/drivers/net/i40e/Makefile index 4fe371d..55b7d31 100644 --- a/drivers/net/i40e/Makefile +++ b/drivers/net/i40e/Makefile @@ -102,6 +102,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_I40E_PMD) += i40e_fdir.c # this lib depends upon: DEPDIRS-$(CONFIG_RTE_LIBRTE_I40E_PMD) += lib/librte_eal lib/librte_ether DEPDIRS-$(CONFIG_RTE_LIBRTE_I40E_PMD) += lib/librte_mempool lib/librte_mbuf -DEPDIRS-$(CONFIG_RTE_LIBRTE_I40E_PMD) += lib/librte_net lib/librte_malloc +DEPDIRS-$(CONFIG_RTE_LIBRTE_I40E_PMD) += lib/librte_net include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/net/ixgbe/Makefile b/drivers/net/ixgbe/Makefile index f92a565..6095cc2 100644 --- a/drivers/net/ixgbe/Makefile +++ b/drivers/net/ixgbe/Makefile @@ -117,7 +117,7 @@ endif # this lib depends upon: DEPDIRS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD) += lib/librte_eal lib/librte_ether DEPDIRS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD) += lib/librte_mempool lib/librte_mbuf -DEPDIRS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD) += lib/librte_net lib/librte_malloc +DEPDIRS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD) += lib/librte_net ifeq ($(CONFIG_RTE_IXGBE_INC_VECTOR)$(CONFIG_RTE_LIBRTE_IXGBE_RX_ALLOW_BULK_ALLOC),yn) $(error The ixgbe vpmd depends on Rx bulk alloc) diff --git a/drivers/net/mlx4/Makefile b/drivers/net/mlx4/Makefile index 725717f..14cb53f 100644 --- a/drivers/net/mlx4/Makefile +++ b/drivers/net/mlx4/Makefile @@ -42,7 +42,6 @@ DEPDIRS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) += lib/librte_ether DEPDIRS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) += lib/librte_mbuf DEPDIRS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) += lib/librte_eal DEPDIRS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) += lib/librte_mempool -DEPDIRS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) += lib/librte_malloc # Basic CFLAGS. CFLAGS += -O3 diff --git a/drivers/net/null/Makefile b/drivers/net/null/Makefile index 6472015..96ba01c 100644 --- a/drivers/net/null/Makefile +++ b/drivers/net/null/Makefile @@ -56,7 +56,6 @@ SYMLINK-y-include += # this lib depends upon: DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_mbuf DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_ether -DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_malloc DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_kvargs include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/net/pcap/Makefile b/drivers/net/pcap/Makefile index 0775dbc..48be913 100644 --- a/drivers/net/pcap/Makefile +++ b/drivers/net/pcap/Makefile @@ -57,7 +57,6 @@ SYMLINK-y-include += # this lib depends upon: DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += lib/librte_mbuf DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += lib/librte_ether -DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += lib/librte_malloc DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += lib/librte_kvargs include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/net/virtio/Makefile b/drivers/net/virtio/Makefile index 21ff7e5..930b60f 100644 --- a/drivers/net/virtio/Makefile +++ b/drivers/net/virtio/Makefile @@ -55,6 +55,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio_ethdev.c # this lib depends upon: DEPDIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += lib/librte_eal lib/librte_ether DEPDIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += lib/librte_mempool lib/librte_mbuf -DEPDIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += lib/librte_net lib/librte_malloc +DEPDIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += lib/librte_net include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/net/vmxnet3/Makefile b/drivers/net/vmxnet3/Makefile index 901cee1..4cf3b33 100644 --- a/drivers/net/vmxnet3/Makefile +++ b/drivers/net/vmxnet3/Makefile @@ -79,6 +79,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += vmxnet3_ethdev.c # this lib depends upon: DEPDIRS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += lib/librte_eal lib/librte_ether DEPDIRS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += lib/librte_mempool lib/librte_mbuf -DEPDIRS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += lib/librte_net lib/librte_malloc +DEPDIRS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += lib/librte_net include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/net/xenvirt/Makefile b/drivers/net/xenvirt/Makefile index f0c796c..9c521d5 100644 --- a/drivers/net/xenvirt/Makefile +++ b/drivers/net/xenvirt/Makefile @@ -56,7 +56,7 @@ SYMLINK-y-include += rte_eth_xenvirt.h # this lib depends upon: DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT) += lib/librte_eal lib/librte_ether DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT) += lib/librte_mempool lib/librte_mbuf -DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT) += lib/librte_net lib/librte_malloc +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT) += lib/librte_net DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT) += lib/librte_cmdline include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/Makefile b/lib/Makefile index 5f480f9..2055539 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -33,7 +33,7 @@ include $(RTE_SDK)/mk/rte.vars.mk DIRS-y += librte_compat DIRS-$(CONFIG_RTE_LIBRTE_EAL) += librte_eal -DIRS-$(CONFIG_RTE_LIBRTE_MALLOC) += librte_malloc +DIRS-$(CONFIG_RTE_LIBRTE_EAL) += librte_malloc DIRS-$(CONFIG_RTE_LIBRTE_RING) += librte_ring DIRS-$(CONFIG_RTE_LIBRTE_MEMPOOL) += librte_mempool DIRS-$(CONFIG_RTE_LIBRTE_MBUF) += librte_mbuf diff --git a/lib/librte_acl/Makefile b/lib/librte_acl/Makefile index 68dc248..46acc2b 100644 --- a/lib/librte_acl/Makefile +++ b/lib/librte_acl/Makefile @@ -75,6 +75,6 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_ACL)-include := rte_acl_osdep.h SYMLINK-$(CONFIG_RTE_LIBRTE_ACL)-include += rte_acl.h # this lib needs eal -DEPDIRS-$(CONFIG_RTE_LIBRTE_ACL) += lib/librte_eal lib/librte_malloc +DEPDIRS-$(CONFIG_RTE_LIBRTE_ACL) += lib/librte_eal include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_eal/bsdapp/eal/Makefile b/lib/librte_eal/bsdapp/eal/Makefile index 40ec648..064b0c5 100644 --- a/lib/librte_eal/bsdapp/eal/Makefile +++ b/lib/librte_eal/bsdapp/eal/Makefile @@ -40,7 +40,6 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include CFLAGS += -I$(RTE_SDK)/lib/librte_ring CFLAGS += -I$(RTE_SDK)/lib/librte_mempool -CFLAGS += -I$(RTE_SDK)/lib/librte_malloc CFLAGS += -I$(RTE_SDK)/lib/librte_ether CFLAGS += -I$(RTE_SDK)/drivers/net/ring CFLAGS += -I$(RTE_SDK)/drivers/net/pcap @@ -79,6 +78,9 @@ SRCS-$(CONFIG_RTE_LIBRTE_EAL_BSDAPP) += eal_common_devargs.c SRCS-$(CONFIG_RTE_LIBRTE_EAL_BSDAPP) += eal_common_dev.c SRCS-$(CONFIG_RTE_LIBRTE_EAL_BSDAPP) += eal_common_options.c SRCS-$(CONFIG_RTE_LIBRTE_EAL_BSDAPP) += eal_common_thread.c +SRCS-$(CONFIG_RTE_LIBRTE_EAL_BSDAPP) += rte_malloc.c +SRCS-$(CONFIG_RTE_LIBRTE_EAL_BSDAPP) += malloc_elem.c +SRCS-$(CONFIG_RTE_LIBRTE_EAL_BSDAPP) += malloc_heap.c CFLAGS_eal.o := -D_GNU_SOURCE #CFLAGS_eal_thread.o := -D_GNU_SOURCE diff --git a/lib/librte_eal/bsdapp/eal/rte_eal_version.map b/lib/librte_eal/bsdapp/eal/rte_eal_version.map index 7e850a9..c107b05 100644 --- a/lib/librte_eal/bsdapp/eal/rte_eal_version.map +++ b/lib/librte_eal/bsdapp/eal/rte_eal_version.map @@ -10,6 +10,8 @@ DPDK_2.0 { pci_driver_list; per_lcore__lcore_id; per_lcore__rte_errno; + rte_calloc; + rte_calloc_socket; rte_cpu_check_supported; rte_cpu_get_flag_enabled; rte_cycles_vmware_tsc_map; @@ -53,6 +55,7 @@ DPDK_2.0 { rte_eal_vdev_uninit; rte_eal_wait_lcore; rte_exit; + rte_free; rte_get_hpet_cycles; rte_get_hpet_hz; rte_get_tsc_hz; @@ -68,6 +71,13 @@ DPDK_2.0 { rte_log_dump_history; rte_log_set_history; rte_logs; + rte_malloc; + rte_malloc_dump_stats; + rte_malloc_get_socket_stats; + rte_malloc_set_limit; + rte_malloc_socket; + rte_malloc_validate; + rte_malloc_virt2phy; rte_mem_lock_page; rte_mem_phy2mch; rte_mem_virt2phy; @@ -81,6 +91,7 @@ DPDK_2.0 { rte_memzone_reserve_bounded; rte_memzone_walk; rte_openlog_stream; + rte_realloc; rte_set_application_usage_hook; rte_set_log_level; rte_set_log_type; @@ -94,6 +105,8 @@ DPDK_2.0 { rte_vlog; rte_xen_dom0_memory_attach; rte_xen_dom0_memory_init; + rte_zmalloc; + rte_zmalloc_socket; test_mp_secondary; local: *; diff --git a/lib/librte_eal/common/Makefile b/lib/librte_eal/common/Makefile index 38772d4..0c43d6a 100644 --- a/lib/librte_eal/common/Makefile +++ b/lib/librte_eal/common/Makefile @@ -40,6 +40,7 @@ INC += rte_string_fns.h rte_version.h INC += rte_eal_memconfig.h rte_malloc_heap.h INC += rte_hexdump.h rte_devargs.h rte_dev.h INC += rte_pci_dev_feature_defs.h rte_pci_dev_features.h +INC += rte_malloc.h ifeq ($(CONFIG_RTE_INSECURE_FUNCTION_WARNING),y) INC += rte_warnings.h diff --git a/lib/librte_eal/common/include/rte_malloc.h b/lib/librte_eal/common/include/rte_malloc.h new file mode 100644 index 0000000..74bb78c --- /dev/null +++ b/lib/librte_eal/common/include/rte_malloc.h @@ -0,0 +1,342 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _RTE_MALLOC_H_ +#define _RTE_MALLOC_H_ + +/** + * @file + * RTE Malloc. This library provides methods for dynamically allocating memory + * from hugepages. + */ + +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * Structure to hold heap statistics obtained from rte_malloc_get_socket_stats function. + */ +struct rte_malloc_socket_stats { + size_t heap_totalsz_bytes; /**< Total bytes on heap */ + size_t heap_freesz_bytes; /**< Total free bytes on heap */ + size_t greatest_free_size; /**< Size in bytes of largest free block */ + unsigned free_count; /**< Number of free elements on heap */ + unsigned alloc_count; /**< Number of allocated elements on heap */ + size_t heap_allocsz_bytes; /**< Total allocated bytes on heap */ +}; + +/** + * This function allocates memory from the huge-page area of memory. The memory + * is not cleared. In NUMA systems, the memory allocated resides on the same + * NUMA socket as the core that calls this function. + * + * @param type + * A string identifying the type of allocated objects (useful for debug + * purposes, such as identifying the cause of a memory leak). Can be NULL. + * @param size + * Size (in bytes) to be allocated. + * @param align + * If 0, the return is a pointer that is suitably aligned for any kind of + * variable (in the same manner as malloc()). + * Otherwise, the return is a pointer that is a multiple of *align*. In + * this case, it must be a power of two. (Minimum alignment is the + * cacheline size, i.e. 64-bytes) + * @return + * - NULL on error. Not enough memory, or invalid arguments (size is 0, + * align is not a power of two). + * - Otherwise, the pointer to the allocated object. + */ +void * +rte_malloc(const char *type, size_t size, unsigned align); + +/** + * Allocate zero'ed memory from the heap. + * + * Equivalent to rte_malloc() except that the memory zone is + * initialised with zeros. In NUMA systems, the memory allocated resides on the + * same NUMA socket as the core that calls this function. + * + * @param type + * A string identifying the type of allocated objects (useful for debug + * purposes, such as identifying the cause of a memory leak). Can be NULL. + * @param size + * Size (in bytes) to be allocated. + * @param align + * If 0, the return is a pointer that is suitably aligned for any kind of + * variable (in the same manner as malloc()). + * Otherwise, the return is a pointer that is a multiple of *align*. In + * this case, it must obviously be a power of two. (Minimum alignment is the + * cacheline size, i.e. 64-bytes) + * @return + * - NULL on error. Not enough memory, or invalid arguments (size is 0, + * align is not a power of two). + * - Otherwise, the pointer to the allocated object. + */ +void * +rte_zmalloc(const char *type, size_t size, unsigned align); + +/** + * Replacement function for calloc(), using huge-page memory. Memory area is + * initialised with zeros. In NUMA systems, the memory allocated resides on the + * same NUMA socket as the core that calls this function. + * + * @param type + * A string identifying the type of allocated objects (useful for debug + * purposes, such as identifying the cause of a memory leak). Can be NULL. + * @param num + * Number of elements to be allocated. + * @param size + * Size (in bytes) of a single element. + * @param align + * If 0, the return is a pointer that is suitably aligned for any kind of + * variable (in the same manner as malloc()). + * Otherwise, the return is a pointer that is a multiple of *align*. In + * this case, it must obviously be a power of two. (Minimum alignment is the + * cacheline size, i.e. 64-bytes) + * @return + * - NULL on error. Not enough memory, or invalid arguments (size is 0, + * align is not a power of two). + * - Otherwise, the pointer to the allocated object. + */ +void * +rte_calloc(const char *type, size_t num, size_t size, unsigned align); + +/** + * Replacement function for realloc(), using huge-page memory. Reserved area + * memory is resized, preserving contents. In NUMA systems, the new area + * resides on the same NUMA socket as the old area. + * + * @param ptr + * Pointer to already allocated memory + * @param size + * Size (in bytes) of new area. If this is 0, memory is freed. + * @param align + * If 0, the return is a pointer that is suitably aligned for any kind of + * variable (in the same manner as malloc()). + * Otherwise, the return is a pointer that is a multiple of *align*. In + * this case, it must obviously be a power of two. (Minimum alignment is the + * cacheline size, i.e. 64-bytes) + * @return + * - NULL on error. Not enough memory, or invalid arguments (size is 0, + * align is not a power of two). + * - Otherwise, the pointer to the reallocated memory. + */ +void * +rte_realloc(void *ptr, size_t size, unsigned align); + +/** + * This function allocates memory from the huge-page area of memory. The memory + * is not cleared. + * + * @param type + * A string identifying the type of allocated objects (useful for debug + * purposes, such as identifying the cause of a memory leak). Can be NULL. + * @param size + * Size (in bytes) to be allocated. + * @param align + * If 0, the return is a pointer that is suitably aligned for any kind of + * variable (in the same manner as malloc()). + * Otherwise, the return is a pointer that is a multiple of *align*. In + * this case, it must be a power of two. (Minimum alignment is the + * cacheline size, i.e. 64-bytes) + * @param socket + * NUMA socket to allocate memory on. If SOCKET_ID_ANY is used, this function + * will behave the same as rte_malloc(). + * @return + * - NULL on error. Not enough memory, or invalid arguments (size is 0, + * align is not a power of two). + * - Otherwise, the pointer to the allocated object. + */ +void * +rte_malloc_socket(const char *type, size_t size, unsigned align, int socket); + +/** + * Allocate zero'ed memory from the heap. + * + * Equivalent to rte_malloc() except that the memory zone is + * initialised with zeros. + * + * @param type + * A string identifying the type of allocated objects (useful for debug + * purposes, such as identifying the cause of a memory leak). Can be NULL. + * @param size + * Size (in bytes) to be allocated. + * @param align + * If 0, the return is a pointer that is suitably aligned for any kind of + * variable (in the same manner as malloc()). + * Otherwise, the return is a pointer that is a multiple of *align*. In + * this case, it must obviously be a power of two. (Minimum alignment is the + * cacheline size, i.e. 64-bytes) + * @param socket + * NUMA socket to allocate memory on. If SOCKET_ID_ANY is used, this function + * will behave the same as rte_zmalloc(). + * @return + * - NULL on error. Not enough memory, or invalid arguments (size is 0, + * align is not a power of two). + * - Otherwise, the pointer to the allocated object. + */ +void * +rte_zmalloc_socket(const char *type, size_t size, unsigned align, int socket); + +/** + * Replacement function for calloc(), using huge-page memory. Memory area is + * initialised with zeros. + * + * @param type + * A string identifying the type of allocated objects (useful for debug + * purposes, such as identifying the cause of a memory leak). Can be NULL. + * @param num + * Number of elements to be allocated. + * @param size + * Size (in bytes) of a single element. + * @param align + * If 0, the return is a pointer that is suitably aligned for any kind of + * variable (in the same manner as malloc()). + * Otherwise, the return is a pointer that is a multiple of *align*. In + * this case, it must obviously be a power of two. (Minimum alignment is the + * cacheline size, i.e. 64-bytes) + * @param socket + * NUMA socket to allocate memory on. If SOCKET_ID_ANY is used, this function + * will behave the same as rte_calloc(). + * @return + * - NULL on error. Not enough memory, or invalid arguments (size is 0, + * align is not a power of two). + * - Otherwise, the pointer to the allocated object. + */ +void * +rte_calloc_socket(const char *type, size_t num, size_t size, unsigned align, int socket); + +/** + * Frees the memory space pointed to by the provided pointer. + * + * This pointer must have been returned by a previous call to + * rte_malloc(), rte_zmalloc(), rte_calloc() or rte_realloc(). The behaviour of + * rte_free() is undefined if the pointer does not match this requirement. + * + * If the pointer is NULL, the function does nothing. + * + * @param ptr + * The pointer to memory to be freed. + */ +void +rte_free(void *ptr); + +/** + * If malloc debug is enabled, check a memory block for header + * and trailer markers to indicate that all is well with the block. + * If size is non-null, also return the size of the block. + * + * @param ptr + * pointer to the start of a data block, must have been returned + * by a previous call to rte_malloc(), rte_zmalloc(), rte_calloc() + * or rte_realloc() + * @param size + * if non-null, and memory block pointer is valid, returns the size + * of the memory block + * @return + * -1 on error, invalid pointer passed or header and trailer markers + * are missing or corrupted + * 0 on success + */ +int +rte_malloc_validate(const void *ptr, size_t *size); + +/** + * Get heap statistics for the specified heap. + * + * @param socket + * An unsigned integer specifying the socket to get heap statistics for + * @param socket_stats + * A structure which provides memory to store statistics + * @return + * Null on error + * Pointer to structure storing statistics on success + */ +int +rte_malloc_get_socket_stats(int socket, + struct rte_malloc_socket_stats *socket_stats); + +/** + * Dump statistics. + * + * Dump for the specified type to the console. If the type argument is + * NULL, all memory types will be dumped. + * + * @param f + * A pointer to a file for output + * @param type + * A string identifying the type of objects to dump, or NULL + * to dump all objects. + */ +void +rte_malloc_dump_stats(FILE *f, const char *type); + +/** + * Set the maximum amount of allocated memory for this type. + * + * This is not yet implemented + * + * @param type + * A string identifying the type of allocated objects. + * @param max + * The maximum amount of allocated bytes for this type. + * @return + * - 0: Success. + * - (-1): Error. + */ +int +rte_malloc_set_limit(const char *type, size_t max); + +/** + * Return the physical address of a virtual address obtained through + * rte_malloc + * + * @param addr + * Adress obtained from a previous rte_malloc call + * @return + * NULL on error + * otherwise return physical address of the buffer + */ +phys_addr_t +rte_malloc_virt2phy(const void *addr); + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_MALLOC_H_ */ diff --git a/lib/librte_eal/common/malloc_elem.c b/lib/librte_eal/common/malloc_elem.c new file mode 100644 index 0000000..a5e1248 --- /dev/null +++ b/lib/librte_eal/common/malloc_elem.c @@ -0,0 +1,320 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "malloc_elem.h" +#include "malloc_heap.h" + +#define MIN_DATA_SIZE (RTE_CACHE_LINE_SIZE) + +/* + * initialise a general malloc_elem header structure + */ +void +malloc_elem_init(struct malloc_elem *elem, + struct malloc_heap *heap, const struct rte_memzone *mz, size_t size) +{ + elem->heap = heap; + elem->mz = mz; + elem->prev = NULL; + memset(&elem->free_list, 0, sizeof(elem->free_list)); + elem->state = ELEM_FREE; + elem->size = size; + elem->pad = 0; + set_header(elem); + set_trailer(elem); +} + +/* + * initialise a dummy malloc_elem header for the end-of-memzone marker + */ +void +malloc_elem_mkend(struct malloc_elem *elem, struct malloc_elem *prev) +{ + malloc_elem_init(elem, prev->heap, prev->mz, 0); + elem->prev = prev; + elem->state = ELEM_BUSY; /* mark busy so its never merged */ +} + +/* + * calculate the starting point of where data of the requested size + * and alignment would fit in the current element. If the data doesn't + * fit, return NULL. + */ +static void * +elem_start_pt(struct malloc_elem *elem, size_t size, unsigned align) +{ + const uintptr_t end_pt = (uintptr_t)elem + + elem->size - MALLOC_ELEM_TRAILER_LEN; + const uintptr_t new_data_start = RTE_ALIGN_FLOOR((end_pt - size), align); + const uintptr_t new_elem_start = new_data_start - MALLOC_ELEM_HEADER_LEN; + + /* if the new start point is before the exist start, it won't fit */ + return (new_elem_start < (uintptr_t)elem) ? NULL : (void *)new_elem_start; +} + +/* + * use elem_start_pt to determine if we get meet the size and + * alignment request from the current element + */ +int +malloc_elem_can_hold(struct malloc_elem *elem, size_t size, unsigned align) +{ + return elem_start_pt(elem, size, align) != NULL; +} + +/* + * split an existing element into two smaller elements at the given + * split_pt parameter. + */ +static void +split_elem(struct malloc_elem *elem, struct malloc_elem *split_pt) +{ + struct malloc_elem *next_elem = RTE_PTR_ADD(elem, elem->size); + const unsigned old_elem_size = (uintptr_t)split_pt - (uintptr_t)elem; + const unsigned new_elem_size = elem->size - old_elem_size; + + malloc_elem_init(split_pt, elem->heap, elem->mz, new_elem_size); + split_pt->prev = elem; + next_elem->prev = split_pt; + elem->size = old_elem_size; + set_trailer(elem); +} + +/* + * Given an element size, compute its freelist index. + * We free an element into the freelist containing similarly-sized elements. + * We try to allocate elements starting with the freelist containing + * similarly-sized elements, and if necessary, we search freelists + * containing larger elements. + * + * Example element size ranges for a heap with five free lists: + * heap->free_head[0] - (0 , 2^8] + * heap->free_head[1] - (2^8 , 2^10] + * heap->free_head[2] - (2^10 ,2^12] + * heap->free_head[3] - (2^12, 2^14] + * heap->free_head[4] - (2^14, MAX_SIZE] + */ +size_t +malloc_elem_free_list_index(size_t size) +{ +#define MALLOC_MINSIZE_LOG2 8 +#define MALLOC_LOG2_INCREMENT 2 + + size_t log2; + size_t index; + + if (size <= (1UL << MALLOC_MINSIZE_LOG2)) + return 0; + + /* Find next power of 2 >= size. */ + log2 = sizeof(size) * 8 - __builtin_clzl(size-1); + + /* Compute freelist index, based on log2(size). */ + index = (log2 - MALLOC_MINSIZE_LOG2 + MALLOC_LOG2_INCREMENT - 1) / + MALLOC_LOG2_INCREMENT; + + return (index <= RTE_HEAP_NUM_FREELISTS-1? + index: RTE_HEAP_NUM_FREELISTS-1); +} + +/* + * Add the specified element to its heap's free list. + */ +void +malloc_elem_free_list_insert(struct malloc_elem *elem) +{ + size_t idx = malloc_elem_free_list_index(elem->size - MALLOC_ELEM_HEADER_LEN); + + elem->state = ELEM_FREE; + LIST_INSERT_HEAD(&elem->heap->free_head[idx], elem, free_list); +} + +/* + * Remove the specified element from its heap's free list. + */ +static void +elem_free_list_remove(struct malloc_elem *elem) +{ + LIST_REMOVE(elem, free_list); +} + +/* + * reserve a block of data in an existing malloc_elem. If the malloc_elem + * is much larger than the data block requested, we split the element in two. + * This function is only called from malloc_heap_alloc so parameter checking + * is not done here, as it's done there previously. + */ +struct malloc_elem * +malloc_elem_alloc(struct malloc_elem *elem, size_t size, unsigned align) +{ + struct malloc_elem *new_elem = elem_start_pt(elem, size, align); + const unsigned old_elem_size = (uintptr_t)new_elem - (uintptr_t)elem; + + if (old_elem_size < MALLOC_ELEM_OVERHEAD + MIN_DATA_SIZE){ + /* don't split it, pad the element instead */ + elem->state = ELEM_BUSY; + elem->pad = old_elem_size; + + /* put a dummy header in padding, to point to real element header */ + if (elem->pad > 0){ /* pad will be at least 64-bytes, as everything + * is cache-line aligned */ + new_elem->pad = elem->pad; + new_elem->state = ELEM_PAD; + new_elem->size = elem->size - elem->pad; + set_header(new_elem); + } + /* remove element from free list */ + elem_free_list_remove(elem); + + return new_elem; + } + + /* we are going to split the element in two. The original element + * remains free, and the new element is the one allocated. + * Re-insert original element, in case its new size makes it + * belong on a different list. + */ + elem_free_list_remove(elem); + split_elem(elem, new_elem); + new_elem->state = ELEM_BUSY; + malloc_elem_free_list_insert(elem); + + return new_elem; +} + +/* + * joing two struct malloc_elem together. elem1 and elem2 must + * be contiguous in memory. + */ +static inline void +join_elem(struct malloc_elem *elem1, struct malloc_elem *elem2) +{ + struct malloc_elem *next = RTE_PTR_ADD(elem2, elem2->size); + elem1->size += elem2->size; + next->prev = elem1; +} + +/* + * free a malloc_elem block by adding it to the free list. If the + * blocks either immediately before or immediately after newly freed block + * are also free, the blocks are merged together. + */ +int +malloc_elem_free(struct malloc_elem *elem) +{ + if (!malloc_elem_cookies_ok(elem) || elem->state != ELEM_BUSY) + return -1; + + rte_spinlock_lock(&(elem->heap->lock)); + struct malloc_elem *next = RTE_PTR_ADD(elem, elem->size); + if (next->state == ELEM_FREE){ + /* remove from free list, join to this one */ + elem_free_list_remove(next); + join_elem(elem, next); + } + + /* check if previous element is free, if so join with it and return, + * need to re-insert in free list, as that element's size is changing + */ + if (elem->prev != NULL && elem->prev->state == ELEM_FREE) { + elem_free_list_remove(elem->prev); + join_elem(elem->prev, elem); + malloc_elem_free_list_insert(elem->prev); + } + /* otherwise add ourselves to the free list */ + else { + malloc_elem_free_list_insert(elem); + elem->pad = 0; + } + /* decrease heap's count of allocated elements */ + elem->heap->alloc_count--; + rte_spinlock_unlock(&(elem->heap->lock)); + + return 0; +} + +/* + * attempt to resize a malloc_elem by expanding into any free space + * immediately after it in memory. + */ +int +malloc_elem_resize(struct malloc_elem *elem, size_t size) +{ + const size_t new_size = size + MALLOC_ELEM_OVERHEAD; + /* if we request a smaller size, then always return ok */ + const size_t current_size = elem->size - elem->pad; + if (current_size >= new_size) + return 0; + + struct malloc_elem *next = RTE_PTR_ADD(elem, elem->size); + rte_spinlock_lock(&elem->heap->lock); + if (next ->state != ELEM_FREE) + goto err_return; + if (current_size + next->size < new_size) + goto err_return; + + /* we now know the element fits, so remove from free list, + * join the two + */ + elem_free_list_remove(next); + join_elem(elem, next); + + if (elem->size - new_size >= MIN_DATA_SIZE + MALLOC_ELEM_OVERHEAD){ + /* now we have a big block together. Lets cut it down a bit, by splitting */ + struct malloc_elem *split_pt = RTE_PTR_ADD(elem, new_size); + split_pt = RTE_PTR_ALIGN_CEIL(split_pt, RTE_CACHE_LINE_SIZE); + split_elem(elem, split_pt); + malloc_elem_free_list_insert(split_pt); + } + rte_spinlock_unlock(&elem->heap->lock); + return 0; + +err_return: + rte_spinlock_unlock(&elem->heap->lock); + return -1; +} diff --git a/lib/librte_eal/common/malloc_elem.h b/lib/librte_eal/common/malloc_elem.h new file mode 100644 index 0000000..9790b1a --- /dev/null +++ b/lib/librte_eal/common/malloc_elem.h @@ -0,0 +1,190 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef MALLOC_ELEM_H_ +#define MALLOC_ELEM_H_ + +#include + +/* dummy definition of struct so we can use pointers to it in malloc_elem struct */ +struct malloc_heap; + +enum elem_state { + ELEM_FREE = 0, + ELEM_BUSY, + ELEM_PAD /* element is a padding-only header */ +}; + +struct malloc_elem { + struct malloc_heap *heap; + struct malloc_elem *volatile prev; /* points to prev elem in memzone */ + LIST_ENTRY(malloc_elem) free_list; /* list of free elements in heap */ + const struct rte_memzone *mz; + volatile enum elem_state state; + uint32_t pad; + size_t size; +#ifdef RTE_LIBRTE_MALLOC_DEBUG + uint64_t header_cookie; /* Cookie marking start of data */ + /* trailer cookie at start + size */ +#endif +} __rte_cache_aligned; + +#ifndef RTE_LIBRTE_MALLOC_DEBUG +static const unsigned MALLOC_ELEM_TRAILER_LEN = 0; + +/* dummy function - just check if pointer is non-null */ +static inline int +malloc_elem_cookies_ok(const struct malloc_elem *elem){ return elem != NULL; } + +/* dummy function - no header if malloc_debug is not enabled */ +static inline void +set_header(struct malloc_elem *elem __rte_unused){ } + +/* dummy function - no trailer if malloc_debug is not enabled */ +static inline void +set_trailer(struct malloc_elem *elem __rte_unused){ } + + +#else +static const unsigned MALLOC_ELEM_TRAILER_LEN = RTE_CACHE_LINE_SIZE; + +#define MALLOC_HEADER_COOKIE 0xbadbadbadadd2e55ULL /**< Header cookie. */ +#define MALLOC_TRAILER_COOKIE 0xadd2e55badbadbadULL /**< Trailer cookie.*/ + +/* define macros to make referencing the header and trailer cookies easier */ +#define MALLOC_ELEM_TRAILER(elem) (*((uint64_t*)RTE_PTR_ADD(elem, \ + elem->size - MALLOC_ELEM_TRAILER_LEN))) +#define MALLOC_ELEM_HEADER(elem) (elem->header_cookie) + +static inline void +set_header(struct malloc_elem *elem) +{ + if (elem != NULL) + MALLOC_ELEM_HEADER(elem) = MALLOC_HEADER_COOKIE; +} + +static inline void +set_trailer(struct malloc_elem *elem) +{ + if (elem != NULL) + MALLOC_ELEM_TRAILER(elem) = MALLOC_TRAILER_COOKIE; +} + +/* check that the header and trailer cookies are set correctly */ +static inline int +malloc_elem_cookies_ok(const struct malloc_elem *elem) +{ + return (elem != NULL && + MALLOC_ELEM_HEADER(elem) == MALLOC_HEADER_COOKIE && + MALLOC_ELEM_TRAILER(elem) == MALLOC_TRAILER_COOKIE); +} + +#endif + +static const unsigned MALLOC_ELEM_HEADER_LEN = sizeof(struct malloc_elem); +#define MALLOC_ELEM_OVERHEAD (MALLOC_ELEM_HEADER_LEN + MALLOC_ELEM_TRAILER_LEN) + +/* + * Given a pointer to the start of a memory block returned by malloc, get + * the actual malloc_elem header for that block. + */ +static inline struct malloc_elem * +malloc_elem_from_data(const void *data) +{ + if (data == NULL) + return NULL; + + struct malloc_elem *elem = RTE_PTR_SUB(data, MALLOC_ELEM_HEADER_LEN); + if (!malloc_elem_cookies_ok(elem)) + return NULL; + return elem->state != ELEM_PAD ? elem: RTE_PTR_SUB(elem, elem->pad); +} + +/* + * initialise a malloc_elem header + */ +void +malloc_elem_init(struct malloc_elem *elem, + struct malloc_heap *heap, + const struct rte_memzone *mz, + size_t size); + +/* + * initialise a dummy malloc_elem header for the end-of-memzone marker + */ +void +malloc_elem_mkend(struct malloc_elem *elem, + struct malloc_elem *prev_free); + +/* + * return true if the current malloc_elem can hold a block of data + * of the requested size and with the requested alignment + */ +int +malloc_elem_can_hold(struct malloc_elem *elem, size_t size, unsigned align); + +/* + * reserve a block of data in an existing malloc_elem. If the malloc_elem + * is much larger than the data block requested, we split the element in two. + */ +struct malloc_elem * +malloc_elem_alloc(struct malloc_elem *elem, size_t size, unsigned align); + +/* + * free a malloc_elem block by adding it to the free list. If the + * blocks either immediately before or immediately after newly freed block + * are also free, the blocks are merged together. + */ +int +malloc_elem_free(struct malloc_elem *elem); + +/* + * attempt to resize a malloc_elem by expanding into any free space + * immediately after it in memory. + */ +int +malloc_elem_resize(struct malloc_elem *elem, size_t size); + +/* + * Given an element size, compute its freelist index. + */ +size_t +malloc_elem_free_list_index(size_t size); + +/* + * Add element to its heap's free list. + */ +void +malloc_elem_free_list_insert(struct malloc_elem *elem); + +#endif /* MALLOC_ELEM_H_ */ diff --git a/lib/librte_eal/common/malloc_heap.c b/lib/librte_eal/common/malloc_heap.c new file mode 100644 index 0000000..8861d27 --- /dev/null +++ b/lib/librte_eal/common/malloc_heap.c @@ -0,0 +1,208 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "malloc_elem.h" +#include "malloc_heap.h" + +/* since the memzone size starts with a digit, it will appear unquoted in + * rte_config.h, so quote it so it can be passed to rte_str_to_size */ +#define MALLOC_MEMZONE_SIZE RTE_STR(RTE_MALLOC_MEMZONE_SIZE) + +/* + * returns the configuration setting for the memzone size as a size_t value + */ +static inline size_t +get_malloc_memzone_size(void) +{ + return rte_str_to_size(MALLOC_MEMZONE_SIZE); +} + +/* + * reserve an extra memory zone and make it available for use by a particular + * heap. This reserves the zone and sets a dummy malloc_elem header at the end + * to prevent overflow. The rest of the zone is added to free list as a single + * large free block + */ +static int +malloc_heap_add_memzone(struct malloc_heap *heap, size_t size, unsigned align) +{ + const unsigned mz_flags = 0; + const size_t block_size = get_malloc_memzone_size(); + /* ensure the data we want to allocate will fit in the memzone */ + const size_t min_size = size + align + MALLOC_ELEM_OVERHEAD * 2; + const struct rte_memzone *mz = NULL; + struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; + unsigned numa_socket = heap - mcfg->malloc_heaps; + + size_t mz_size = min_size; + if (mz_size < block_size) + mz_size = block_size; + + char mz_name[RTE_MEMZONE_NAMESIZE]; + snprintf(mz_name, sizeof(mz_name), "MALLOC_S%u_HEAP_%u", + numa_socket, heap->mz_count++); + + /* try getting a block. if we fail and we don't need as big a block + * as given in the config, we can shrink our request and try again + */ + do { + mz = rte_memzone_reserve(mz_name, mz_size, numa_socket, + mz_flags); + if (mz == NULL) + mz_size /= 2; + } while (mz == NULL && mz_size > min_size); + if (mz == NULL) + return -1; + + /* allocate the memory block headers, one at end, one at start */ + struct malloc_elem *start_elem = (struct malloc_elem *)mz->addr; + struct malloc_elem *end_elem = RTE_PTR_ADD(mz->addr, + mz_size - MALLOC_ELEM_OVERHEAD); + end_elem = RTE_PTR_ALIGN_FLOOR(end_elem, RTE_CACHE_LINE_SIZE); + + const unsigned elem_size = (uintptr_t)end_elem - (uintptr_t)start_elem; + malloc_elem_init(start_elem, heap, mz, elem_size); + malloc_elem_mkend(end_elem, start_elem); + malloc_elem_free_list_insert(start_elem); + + /* increase heap total size by size of new memzone */ + heap->total_size+=mz_size - MALLOC_ELEM_OVERHEAD; + return 0; +} + +/* + * Iterates through the freelist for a heap to find a free element + * which can store data of the required size and with the requested alignment. + * Returns null on failure, or pointer to element on success. + */ +static struct malloc_elem * +find_suitable_element(struct malloc_heap *heap, size_t size, unsigned align) +{ + size_t idx; + struct malloc_elem *elem; + + for (idx = malloc_elem_free_list_index(size); + idx < RTE_HEAP_NUM_FREELISTS; idx++) + { + for (elem = LIST_FIRST(&heap->free_head[idx]); + !!elem; elem = LIST_NEXT(elem, free_list)) + { + if (malloc_elem_can_hold(elem, size, align)) + return elem; + } + } + return NULL; +} + +/* + * Main function called by malloc to allocate a block of memory from the + * heap. It locks the free list, scans it, and adds a new memzone if the + * scan fails. Once the new memzone is added, it re-scans and should return + * the new element after releasing the lock. + */ +void * +malloc_heap_alloc(struct malloc_heap *heap, + const char *type __attribute__((unused)), size_t size, unsigned align) +{ + size = RTE_CACHE_LINE_ROUNDUP(size); + align = RTE_CACHE_LINE_ROUNDUP(align); + rte_spinlock_lock(&heap->lock); + struct malloc_elem *elem = find_suitable_element(heap, size, align); + if (elem == NULL){ + if ((malloc_heap_add_memzone(heap, size, align)) == 0) + elem = find_suitable_element(heap, size, align); + } + + if (elem != NULL){ + elem = malloc_elem_alloc(elem, size, align); + /* increase heap's count of allocated elements */ + heap->alloc_count++; + } + rte_spinlock_unlock(&heap->lock); + return elem == NULL ? NULL : (void *)(&elem[1]); + +} + +/* + * Function to retrieve data for heap on given socket + */ +int +malloc_heap_get_stats(const struct malloc_heap *heap, + struct rte_malloc_socket_stats *socket_stats) +{ + size_t idx; + struct malloc_elem *elem; + + /* Initialise variables for heap */ + socket_stats->free_count = 0; + socket_stats->heap_freesz_bytes = 0; + socket_stats->greatest_free_size = 0; + + /* Iterate through free list */ + for (idx = 0; idx < RTE_HEAP_NUM_FREELISTS; idx++) { + for (elem = LIST_FIRST(&heap->free_head[idx]); + !!elem; elem = LIST_NEXT(elem, free_list)) + { + socket_stats->free_count++; + socket_stats->heap_freesz_bytes += elem->size; + if (elem->size > socket_stats->greatest_free_size) + socket_stats->greatest_free_size = elem->size; + } + } + /* Get stats on overall heap and allocated memory on this heap */ + socket_stats->heap_totalsz_bytes = heap->total_size; + socket_stats->heap_allocsz_bytes = (socket_stats->heap_totalsz_bytes - + socket_stats->heap_freesz_bytes); + socket_stats->alloc_count = heap->alloc_count; + return 0; +} diff --git a/lib/librte_eal/common/malloc_heap.h b/lib/librte_eal/common/malloc_heap.h new file mode 100644 index 0000000..a47136d --- /dev/null +++ b/lib/librte_eal/common/malloc_heap.h @@ -0,0 +1,70 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef MALLOC_HEAP_H_ +#define MALLOC_HEAP_H_ + +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +static inline unsigned +malloc_get_numa_socket(void) +{ + unsigned socket_id = rte_socket_id(); + + if (socket_id == (unsigned)SOCKET_ID_ANY) + return 0; + + return socket_id; +} + +void * +malloc_heap_alloc(struct malloc_heap *heap, const char *type, + size_t size, unsigned align); + +int +malloc_heap_get_stats(const struct malloc_heap *heap, + struct rte_malloc_socket_stats *socket_stats); + +int +rte_eal_heap_memzone_init(void); + +#ifdef __cplusplus +} +#endif + +#endif /* MALLOC_HEAP_H_ */ diff --git a/lib/librte_eal/common/rte_malloc.c b/lib/librte_eal/common/rte_malloc.c new file mode 100644 index 0000000..c313a57 --- /dev/null +++ b/lib/librte_eal/common/rte_malloc.c @@ -0,0 +1,260 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include "malloc_elem.h" +#include "malloc_heap.h" + + +/* Free the memory space back to heap */ +void rte_free(void *addr) +{ + if (addr == NULL) return; + if (malloc_elem_free(malloc_elem_from_data(addr)) < 0) + rte_panic("Fatal error: Invalid memory\n"); +} + +/* + * Allocate memory on specified heap. + */ +void * +rte_malloc_socket(const char *type, size_t size, unsigned align, int socket_arg) +{ + struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; + int socket, i; + void *ret; + + /* return NULL if size is 0 or alignment is not power-of-2 */ + if (size == 0 || (align && !rte_is_power_of_2(align))) + return NULL; + + if (socket_arg == SOCKET_ID_ANY) + socket = malloc_get_numa_socket(); + else + socket = socket_arg; + + /* Check socket parameter */ + if (socket >= RTE_MAX_NUMA_NODES) + return NULL; + + ret = malloc_heap_alloc(&mcfg->malloc_heaps[socket], type, + size, align == 0 ? 1 : align); + if (ret != NULL || socket_arg != SOCKET_ID_ANY) + return ret; + + /* try other heaps */ + for (i = 0; i < RTE_MAX_NUMA_NODES; i++) { + /* we already tried this one */ + if (i == socket) + continue; + + ret = malloc_heap_alloc(&mcfg->malloc_heaps[i], type, + size, align == 0 ? 1 : align); + if (ret != NULL) + return ret; + } + + return NULL; +} + +/* + * Allocate memory on default heap. + */ +void * +rte_malloc(const char *type, size_t size, unsigned align) +{ + return rte_malloc_socket(type, size, align, SOCKET_ID_ANY); +} + +/* + * Allocate zero'd memory on specified heap. + */ +void * +rte_zmalloc_socket(const char *type, size_t size, unsigned align, int socket) +{ + void *ptr = rte_malloc_socket(type, size, align, socket); + + if (ptr != NULL) + memset(ptr, 0, size); + return ptr; +} + +/* + * Allocate zero'd memory on default heap. + */ +void * +rte_zmalloc(const char *type, size_t size, unsigned align) +{ + return rte_zmalloc_socket(type, size, align, SOCKET_ID_ANY); +} + +/* + * Allocate zero'd memory on specified heap. + */ +void * +rte_calloc_socket(const char *type, size_t num, size_t size, unsigned align, int socket) +{ + return rte_zmalloc_socket(type, num * size, align, socket); +} + +/* + * Allocate zero'd memory on default heap. + */ +void * +rte_calloc(const char *type, size_t num, size_t size, unsigned align) +{ + return rte_zmalloc(type, num * size, align); +} + +/* + * Resize allocated memory. + */ +void * +rte_realloc(void *ptr, size_t size, unsigned align) +{ + if (ptr == NULL) + return rte_malloc(NULL, size, align); + + struct malloc_elem *elem = malloc_elem_from_data(ptr); + if (elem == NULL) + rte_panic("Fatal error: memory corruption detected\n"); + + size = RTE_CACHE_LINE_ROUNDUP(size), align = RTE_CACHE_LINE_ROUNDUP(align); + /* check alignment matches first, and if ok, see if we can resize block */ + if (RTE_PTR_ALIGN(ptr,align) == ptr && + malloc_elem_resize(elem, size) == 0) + return ptr; + + /* either alignment is off, or we have no room to expand, + * so move data. */ + void *new_ptr = rte_malloc(NULL, size, align); + if (new_ptr == NULL) + return NULL; + const unsigned old_size = elem->size - MALLOC_ELEM_OVERHEAD; + rte_memcpy(new_ptr, ptr, old_size < size ? old_size : size); + rte_free(ptr); + + return new_ptr; +} + +int +rte_malloc_validate(const void *ptr, size_t *size) +{ + const struct malloc_elem *elem = malloc_elem_from_data(ptr); + if (!malloc_elem_cookies_ok(elem)) + return -1; + if (size != NULL) + *size = elem->size - elem->pad - MALLOC_ELEM_OVERHEAD; + return 0; +} + +/* + * Function to retrieve data for heap on given socket + */ +int +rte_malloc_get_socket_stats(int socket, + struct rte_malloc_socket_stats *socket_stats) +{ + struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; + + if (socket >= RTE_MAX_NUMA_NODES || socket < 0) + return -1; + + return malloc_heap_get_stats(&mcfg->malloc_heaps[socket], socket_stats); +} + +/* + * Print stats on memory type. If type is NULL, info on all types is printed + */ +void +rte_malloc_dump_stats(FILE *f, __rte_unused const char *type) +{ + unsigned int socket; + struct rte_malloc_socket_stats sock_stats; + /* Iterate through all initialised heaps */ + for (socket=0; socket< RTE_MAX_NUMA_NODES; socket++) { + if ((rte_malloc_get_socket_stats(socket, &sock_stats) < 0)) + continue; + + fprintf(f, "Socket:%u\n", socket); + fprintf(f, "\tHeap_size:%zu,\n", sock_stats.heap_totalsz_bytes); + fprintf(f, "\tFree_size:%zu,\n", sock_stats.heap_freesz_bytes); + fprintf(f, "\tAlloc_size:%zu,\n", sock_stats.heap_allocsz_bytes); + fprintf(f, "\tGreatest_free_size:%zu,\n", + sock_stats.greatest_free_size); + fprintf(f, "\tAlloc_count:%u,\n",sock_stats.alloc_count); + fprintf(f, "\tFree_count:%u,\n", sock_stats.free_count); + } + return; +} + +/* + * TODO: Set limit to memory that can be allocated to memory type + */ +int +rte_malloc_set_limit(__rte_unused const char *type, + __rte_unused size_t max) +{ + return 0; +} + +/* + * Return the physical address of a virtual address obtained through rte_malloc + */ +phys_addr_t +rte_malloc_virt2phy(const void *addr) +{ + const struct malloc_elem *elem = malloc_elem_from_data(addr); + if (elem == NULL) + return 0; + return elem->mz->phys_addr + ((uintptr_t)addr - (uintptr_t)elem->mz->addr); +} diff --git a/lib/librte_eal/linuxapp/eal/Makefile b/lib/librte_eal/linuxapp/eal/Makefile index 42a16fe..00ed62e 100644 --- a/lib/librte_eal/linuxapp/eal/Makefile +++ b/lib/librte_eal/linuxapp/eal/Makefile @@ -44,7 +44,6 @@ CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common CFLAGS += -I$(RTE_SDK)/lib/librte_eal/common/include CFLAGS += -I$(RTE_SDK)/lib/librte_ring CFLAGS += -I$(RTE_SDK)/lib/librte_mempool -CFLAGS += -I$(RTE_SDK)/lib/librte_malloc CFLAGS += -I$(RTE_SDK)/lib/librte_ether CFLAGS += -I$(RTE_SDK)/lib/librte_ivshmem CFLAGS += -I$(RTE_SDK)/drivers/net/ring @@ -91,6 +90,9 @@ SRCS-$(CONFIG_RTE_LIBRTE_EAL_LINUXAPP) += eal_common_devargs.c SRCS-$(CONFIG_RTE_LIBRTE_EAL_LINUXAPP) += eal_common_dev.c SRCS-$(CONFIG_RTE_LIBRTE_EAL_LINUXAPP) += eal_common_options.c SRCS-$(CONFIG_RTE_LIBRTE_EAL_LINUXAPP) += eal_common_thread.c +SRCS-$(CONFIG_RTE_LIBRTE_EAL_LINUXAPP) += rte_malloc.c +SRCS-$(CONFIG_RTE_LIBRTE_EAL_LINUXAPP) += malloc_elem.c +SRCS-$(CONFIG_RTE_LIBRTE_EAL_LINUXAPP) += malloc_heap.c CFLAGS_eal.o := -D_GNU_SOURCE CFLAGS_eal_interrupts.o := -D_GNU_SOURCE diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map index 7e850a9..c107b05 100644 --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map @@ -10,6 +10,8 @@ DPDK_2.0 { pci_driver_list; per_lcore__lcore_id; per_lcore__rte_errno; + rte_calloc; + rte_calloc_socket; rte_cpu_check_supported; rte_cpu_get_flag_enabled; rte_cycles_vmware_tsc_map; @@ -53,6 +55,7 @@ DPDK_2.0 { rte_eal_vdev_uninit; rte_eal_wait_lcore; rte_exit; + rte_free; rte_get_hpet_cycles; rte_get_hpet_hz; rte_get_tsc_hz; @@ -68,6 +71,13 @@ DPDK_2.0 { rte_log_dump_history; rte_log_set_history; rte_logs; + rte_malloc; + rte_malloc_dump_stats; + rte_malloc_get_socket_stats; + rte_malloc_set_limit; + rte_malloc_socket; + rte_malloc_validate; + rte_malloc_virt2phy; rte_mem_lock_page; rte_mem_phy2mch; rte_mem_virt2phy; @@ -81,6 +91,7 @@ DPDK_2.0 { rte_memzone_reserve_bounded; rte_memzone_walk; rte_openlog_stream; + rte_realloc; rte_set_application_usage_hook; rte_set_log_level; rte_set_log_type; @@ -94,6 +105,8 @@ DPDK_2.0 { rte_vlog; rte_xen_dom0_memory_attach; rte_xen_dom0_memory_init; + rte_zmalloc; + rte_zmalloc_socket; test_mp_secondary; local: *; diff --git a/lib/librte_hash/Makefile b/lib/librte_hash/Makefile index 039da46..4bb3848 100644 --- a/lib/librte_hash/Makefile +++ b/lib/librte_hash/Makefile @@ -53,6 +53,6 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_HASH)-include += rte_thash.h SYMLINK-$(CONFIG_RTE_LIBRTE_HASH)-include += rte_fbk_hash.h # this lib needs eal and ring -DEPDIRS-$(CONFIG_RTE_LIBRTE_HASH) += lib/librte_eal lib/librte_malloc lib/librte_ring +DEPDIRS-$(CONFIG_RTE_LIBRTE_HASH) += lib/librte_eal lib/librte_ring include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_lpm/Makefile b/lib/librte_lpm/Makefile index 35e6389..0a7a888 100644 --- a/lib/librte_lpm/Makefile +++ b/lib/librte_lpm/Makefile @@ -48,6 +48,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_LPM) := rte_lpm.c rte_lpm6.c SYMLINK-$(CONFIG_RTE_LIBRTE_LPM)-include := rte_lpm.h rte_lpm6.h # this lib needs eal -DEPDIRS-$(CONFIG_RTE_LIBRTE_LPM) += lib/librte_eal lib/librte_malloc +DEPDIRS-$(CONFIG_RTE_LIBRTE_LPM) += lib/librte_eal include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_malloc/Makefile b/lib/librte_malloc/Makefile index 947e41c..32d86b9 100644 --- a/lib/librte_malloc/Makefile +++ b/lib/librte_malloc/Makefile @@ -28,7 +28,6 @@ # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - include $(RTE_SDK)/mk/rte.vars.mk # library name @@ -41,10 +40,7 @@ CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 EXPORT_MAP := rte_malloc_version.map # all source are stored in SRCS-y -SRCS-$(CONFIG_RTE_LIBRTE_MALLOC) := rte_malloc.c malloc_elem.c malloc_heap.c - -# install includes -SYMLINK-$(CONFIG_RTE_LIBRTE_MALLOC)-include := rte_malloc.h +SRCS-$(CONFIG_RTE_LIBRTE_MALLOC) := rte_malloc.c # this lib needs eal DEPDIRS-$(CONFIG_RTE_LIBRTE_MALLOC) += lib/librte_eal diff --git a/lib/librte_malloc/malloc_elem.c b/lib/librte_malloc/malloc_elem.c deleted file mode 100644 index a5e1248..0000000 --- a/lib/librte_malloc/malloc_elem.c +++ /dev/null @@ -1,320 +0,0 @@ -/*- - * BSD LICENSE - * - * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in - * the documentation and/or other materials provided with the - * distribution. - * * Neither the name of Intel Corporation nor the names of its - * contributors may be used to endorse or promote products derived - * from this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - */ -#include -#include -#include -#include -#include - -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include "malloc_elem.h" -#include "malloc_heap.h" - -#define MIN_DATA_SIZE (RTE_CACHE_LINE_SIZE) - -/* - * initialise a general malloc_elem header structure - */ -void -malloc_elem_init(struct malloc_elem *elem, - struct malloc_heap *heap, const struct rte_memzone *mz, size_t size) -{ - elem->heap = heap; - elem->mz = mz; - elem->prev = NULL; - memset(&elem->free_list, 0, sizeof(elem->free_list)); - elem->state = ELEM_FREE; - elem->size = size; - elem->pad = 0; - set_header(elem); - set_trailer(elem); -} - -/* - * initialise a dummy malloc_elem header for the end-of-memzone marker - */ -void -malloc_elem_mkend(struct malloc_elem *elem, struct malloc_elem *prev) -{ - malloc_elem_init(elem, prev->heap, prev->mz, 0); - elem->prev = prev; - elem->state = ELEM_BUSY; /* mark busy so its never merged */ -} - -/* - * calculate the starting point of where data of the requested size - * and alignment would fit in the current element. If the data doesn't - * fit, return NULL. - */ -static void * -elem_start_pt(struct malloc_elem *elem, size_t size, unsigned align) -{ - const uintptr_t end_pt = (uintptr_t)elem + - elem->size - MALLOC_ELEM_TRAILER_LEN; - const uintptr_t new_data_start = RTE_ALIGN_FLOOR((end_pt - size), align); - const uintptr_t new_elem_start = new_data_start - MALLOC_ELEM_HEADER_LEN; - - /* if the new start point is before the exist start, it won't fit */ - return (new_elem_start < (uintptr_t)elem) ? NULL : (void *)new_elem_start; -} - -/* - * use elem_start_pt to determine if we get meet the size and - * alignment request from the current element - */ -int -malloc_elem_can_hold(struct malloc_elem *elem, size_t size, unsigned align) -{ - return elem_start_pt(elem, size, align) != NULL; -} - -/* - * split an existing element into two smaller elements at the given - * split_pt parameter. - */ -static void -split_elem(struct malloc_elem *elem, struct malloc_elem *split_pt) -{ - struct malloc_elem *next_elem = RTE_PTR_ADD(elem, elem->size); - const unsigned old_elem_size = (uintptr_t)split_pt - (uintptr_t)elem; - const unsigned new_elem_size = elem->size - old_elem_size; - - malloc_elem_init(split_pt, elem->heap, elem->mz, new_elem_size); - split_pt->prev = elem; - next_elem->prev = split_pt; - elem->size = old_elem_size; - set_trailer(elem); -} - -/* - * Given an element size, compute its freelist index. - * We free an element into the freelist containing similarly-sized elements. - * We try to allocate elements starting with the freelist containing - * similarly-sized elements, and if necessary, we search freelists - * containing larger elements. - * - * Example element size ranges for a heap with five free lists: - * heap->free_head[0] - (0 , 2^8] - * heap->free_head[1] - (2^8 , 2^10] - * heap->free_head[2] - (2^10 ,2^12] - * heap->free_head[3] - (2^12, 2^14] - * heap->free_head[4] - (2^14, MAX_SIZE] - */ -size_t -malloc_elem_free_list_index(size_t size) -{ -#define MALLOC_MINSIZE_LOG2 8 -#define MALLOC_LOG2_INCREMENT 2 - - size_t log2; - size_t index; - - if (size <= (1UL << MALLOC_MINSIZE_LOG2)) - return 0; - - /* Find next power of 2 >= size. */ - log2 = sizeof(size) * 8 - __builtin_clzl(size-1); - - /* Compute freelist index, based on log2(size). */ - index = (log2 - MALLOC_MINSIZE_LOG2 + MALLOC_LOG2_INCREMENT - 1) / - MALLOC_LOG2_INCREMENT; - - return (index <= RTE_HEAP_NUM_FREELISTS-1? - index: RTE_HEAP_NUM_FREELISTS-1); -} - -/* - * Add the specified element to its heap's free list. - */ -void -malloc_elem_free_list_insert(struct malloc_elem *elem) -{ - size_t idx = malloc_elem_free_list_index(elem->size - MALLOC_ELEM_HEADER_LEN); - - elem->state = ELEM_FREE; - LIST_INSERT_HEAD(&elem->heap->free_head[idx], elem, free_list); -} - -/* - * Remove the specified element from its heap's free list. - */ -static void -elem_free_list_remove(struct malloc_elem *elem) -{ - LIST_REMOVE(elem, free_list); -} - -/* - * reserve a block of data in an existing malloc_elem. If the malloc_elem - * is much larger than the data block requested, we split the element in two. - * This function is only called from malloc_heap_alloc so parameter checking - * is not done here, as it's done there previously. - */ -struct malloc_elem * -malloc_elem_alloc(struct malloc_elem *elem, size_t size, unsigned align) -{ - struct malloc_elem *new_elem = elem_start_pt(elem, size, align); - const unsigned old_elem_size = (uintptr_t)new_elem - (uintptr_t)elem; - - if (old_elem_size < MALLOC_ELEM_OVERHEAD + MIN_DATA_SIZE){ - /* don't split it, pad the element instead */ - elem->state = ELEM_BUSY; - elem->pad = old_elem_size; - - /* put a dummy header in padding, to point to real element header */ - if (elem->pad > 0){ /* pad will be at least 64-bytes, as everything - * is cache-line aligned */ - new_elem->pad = elem->pad; - new_elem->state = ELEM_PAD; - new_elem->size = elem->size - elem->pad; - set_header(new_elem); - } - /* remove element from free list */ - elem_free_list_remove(elem); - - return new_elem; - } - - /* we are going to split the element in two. The original element - * remains free, and the new element is the one allocated. - * Re-insert original element, in case its new size makes it - * belong on a different list. - */ - elem_free_list_remove(elem); - split_elem(elem, new_elem); - new_elem->state = ELEM_BUSY; - malloc_elem_free_list_insert(elem); - - return new_elem; -} - -/* - * joing two struct malloc_elem together. elem1 and elem2 must - * be contiguous in memory. - */ -static inline void -join_elem(struct malloc_elem *elem1, struct malloc_elem *elem2) -{ - struct malloc_elem *next = RTE_PTR_ADD(elem2, elem2->size); - elem1->size += elem2->size; - next->prev = elem1; -} - -/* - * free a malloc_elem block by adding it to the free list. If the - * blocks either immediately before or immediately after newly freed block - * are also free, the blocks are merged together. - */ -int -malloc_elem_free(struct malloc_elem *elem) -{ - if (!malloc_elem_cookies_ok(elem) || elem->state != ELEM_BUSY) - return -1; - - rte_spinlock_lock(&(elem->heap->lock)); - struct malloc_elem *next = RTE_PTR_ADD(elem, elem->size); - if (next->state == ELEM_FREE){ - /* remove from free list, join to this one */ - elem_free_list_remove(next); - join_elem(elem, next); - } - - /* check if previous element is free, if so join with it and return, - * need to re-insert in free list, as that element's size is changing - */ - if (elem->prev != NULL && elem->prev->state == ELEM_FREE) { - elem_free_list_remove(elem->prev); - join_elem(elem->prev, elem); - malloc_elem_free_list_insert(elem->prev); - } - /* otherwise add ourselves to the free list */ - else { - malloc_elem_free_list_insert(elem); - elem->pad = 0; - } - /* decrease heap's count of allocated elements */ - elem->heap->alloc_count--; - rte_spinlock_unlock(&(elem->heap->lock)); - - return 0; -} - -/* - * attempt to resize a malloc_elem by expanding into any free space - * immediately after it in memory. - */ -int -malloc_elem_resize(struct malloc_elem *elem, size_t size) -{ - const size_t new_size = size + MALLOC_ELEM_OVERHEAD; - /* if we request a smaller size, then always return ok */ - const size_t current_size = elem->size - elem->pad; - if (current_size >= new_size) - return 0; - - struct malloc_elem *next = RTE_PTR_ADD(elem, elem->size); - rte_spinlock_lock(&elem->heap->lock); - if (next ->state != ELEM_FREE) - goto err_return; - if (current_size + next->size < new_size) - goto err_return; - - /* we now know the element fits, so remove from free list, - * join the two - */ - elem_free_list_remove(next); - join_elem(elem, next); - - if (elem->size - new_size >= MIN_DATA_SIZE + MALLOC_ELEM_OVERHEAD){ - /* now we have a big block together. Lets cut it down a bit, by splitting */ - struct malloc_elem *split_pt = RTE_PTR_ADD(elem, new_size); - split_pt = RTE_PTR_ALIGN_CEIL(split_pt, RTE_CACHE_LINE_SIZE); - split_elem(elem, split_pt); - malloc_elem_free_list_insert(split_pt); - } - rte_spinlock_unlock(&elem->heap->lock); - return 0; - -err_return: - rte_spinlock_unlock(&elem->heap->lock); - return -1; -} diff --git a/lib/librte_malloc/malloc_elem.h b/lib/librte_malloc/malloc_elem.h deleted file mode 100644 index 9790b1a..0000000 --- a/lib/librte_malloc/malloc_elem.h +++ /dev/null @@ -1,190 +0,0 @@ -/*- - * BSD LICENSE - * - * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in - * the documentation and/or other materials provided with the - * distribution. - * * Neither the name of Intel Corporation nor the names of its - * contributors may be used to endorse or promote products derived - * from this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - */ - -#ifndef MALLOC_ELEM_H_ -#define MALLOC_ELEM_H_ - -#include - -/* dummy definition of struct so we can use pointers to it in malloc_elem struct */ -struct malloc_heap; - -enum elem_state { - ELEM_FREE = 0, - ELEM_BUSY, - ELEM_PAD /* element is a padding-only header */ -}; - -struct malloc_elem { - struct malloc_heap *heap; - struct malloc_elem *volatile prev; /* points to prev elem in memzone */ - LIST_ENTRY(malloc_elem) free_list; /* list of free elements in heap */ - const struct rte_memzone *mz; - volatile enum elem_state state; - uint32_t pad; - size_t size; -#ifdef RTE_LIBRTE_MALLOC_DEBUG - uint64_t header_cookie; /* Cookie marking start of data */ - /* trailer cookie at start + size */ -#endif -} __rte_cache_aligned; - -#ifndef RTE_LIBRTE_MALLOC_DEBUG -static const unsigned MALLOC_ELEM_TRAILER_LEN = 0; - -/* dummy function - just check if pointer is non-null */ -static inline int -malloc_elem_cookies_ok(const struct malloc_elem *elem){ return elem != NULL; } - -/* dummy function - no header if malloc_debug is not enabled */ -static inline void -set_header(struct malloc_elem *elem __rte_unused){ } - -/* dummy function - no trailer if malloc_debug is not enabled */ -static inline void -set_trailer(struct malloc_elem *elem __rte_unused){ } - - -#else -static const unsigned MALLOC_ELEM_TRAILER_LEN = RTE_CACHE_LINE_SIZE; - -#define MALLOC_HEADER_COOKIE 0xbadbadbadadd2e55ULL /**< Header cookie. */ -#define MALLOC_TRAILER_COOKIE 0xadd2e55badbadbadULL /**< Trailer cookie.*/ - -/* define macros to make referencing the header and trailer cookies easier */ -#define MALLOC_ELEM_TRAILER(elem) (*((uint64_t*)RTE_PTR_ADD(elem, \ - elem->size - MALLOC_ELEM_TRAILER_LEN))) -#define MALLOC_ELEM_HEADER(elem) (elem->header_cookie) - -static inline void -set_header(struct malloc_elem *elem) -{ - if (elem != NULL) - MALLOC_ELEM_HEADER(elem) = MALLOC_HEADER_COOKIE; -} - -static inline void -set_trailer(struct malloc_elem *elem) -{ - if (elem != NULL) - MALLOC_ELEM_TRAILER(elem) = MALLOC_TRAILER_COOKIE; -} - -/* check that the header and trailer cookies are set correctly */ -static inline int -malloc_elem_cookies_ok(const struct malloc_elem *elem) -{ - return (elem != NULL && - MALLOC_ELEM_HEADER(elem) == MALLOC_HEADER_COOKIE && - MALLOC_ELEM_TRAILER(elem) == MALLOC_TRAILER_COOKIE); -} - -#endif - -static const unsigned MALLOC_ELEM_HEADER_LEN = sizeof(struct malloc_elem); -#define MALLOC_ELEM_OVERHEAD (MALLOC_ELEM_HEADER_LEN + MALLOC_ELEM_TRAILER_LEN) - -/* - * Given a pointer to the start of a memory block returned by malloc, get - * the actual malloc_elem header for that block. - */ -static inline struct malloc_elem * -malloc_elem_from_data(const void *data) -{ - if (data == NULL) - return NULL; - - struct malloc_elem *elem = RTE_PTR_SUB(data, MALLOC_ELEM_HEADER_LEN); - if (!malloc_elem_cookies_ok(elem)) - return NULL; - return elem->state != ELEM_PAD ? elem: RTE_PTR_SUB(elem, elem->pad); -} - -/* - * initialise a malloc_elem header - */ -void -malloc_elem_init(struct malloc_elem *elem, - struct malloc_heap *heap, - const struct rte_memzone *mz, - size_t size); - -/* - * initialise a dummy malloc_elem header for the end-of-memzone marker - */ -void -malloc_elem_mkend(struct malloc_elem *elem, - struct malloc_elem *prev_free); - -/* - * return true if the current malloc_elem can hold a block of data - * of the requested size and with the requested alignment - */ -int -malloc_elem_can_hold(struct malloc_elem *elem, size_t size, unsigned align); - -/* - * reserve a block of data in an existing malloc_elem. If the malloc_elem - * is much larger than the data block requested, we split the element in two. - */ -struct malloc_elem * -malloc_elem_alloc(struct malloc_elem *elem, size_t size, unsigned align); - -/* - * free a malloc_elem block by adding it to the free list. If the - * blocks either immediately before or immediately after newly freed block - * are also free, the blocks are merged together. - */ -int -malloc_elem_free(struct malloc_elem *elem); - -/* - * attempt to resize a malloc_elem by expanding into any free space - * immediately after it in memory. - */ -int -malloc_elem_resize(struct malloc_elem *elem, size_t size); - -/* - * Given an element size, compute its freelist index. - */ -size_t -malloc_elem_free_list_index(size_t size); - -/* - * Add element to its heap's free list. - */ -void -malloc_elem_free_list_insert(struct malloc_elem *elem); - -#endif /* MALLOC_ELEM_H_ */ diff --git a/lib/librte_malloc/malloc_heap.c b/lib/librte_malloc/malloc_heap.c deleted file mode 100644 index 8861d27..0000000 --- a/lib/librte_malloc/malloc_heap.c +++ /dev/null @@ -1,208 +0,0 @@ -/*- - * BSD LICENSE - * - * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in - * the documentation and/or other materials provided with the - * distribution. - * * Neither the name of Intel Corporation nor the names of its - * contributors may be used to endorse or promote products derived - * from this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - */ -#include -#include -#include -#include -#include -#include -#include - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include "malloc_elem.h" -#include "malloc_heap.h" - -/* since the memzone size starts with a digit, it will appear unquoted in - * rte_config.h, so quote it so it can be passed to rte_str_to_size */ -#define MALLOC_MEMZONE_SIZE RTE_STR(RTE_MALLOC_MEMZONE_SIZE) - -/* - * returns the configuration setting for the memzone size as a size_t value - */ -static inline size_t -get_malloc_memzone_size(void) -{ - return rte_str_to_size(MALLOC_MEMZONE_SIZE); -} - -/* - * reserve an extra memory zone and make it available for use by a particular - * heap. This reserves the zone and sets a dummy malloc_elem header at the end - * to prevent overflow. The rest of the zone is added to free list as a single - * large free block - */ -static int -malloc_heap_add_memzone(struct malloc_heap *heap, size_t size, unsigned align) -{ - const unsigned mz_flags = 0; - const size_t block_size = get_malloc_memzone_size(); - /* ensure the data we want to allocate will fit in the memzone */ - const size_t min_size = size + align + MALLOC_ELEM_OVERHEAD * 2; - const struct rte_memzone *mz = NULL; - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; - unsigned numa_socket = heap - mcfg->malloc_heaps; - - size_t mz_size = min_size; - if (mz_size < block_size) - mz_size = block_size; - - char mz_name[RTE_MEMZONE_NAMESIZE]; - snprintf(mz_name, sizeof(mz_name), "MALLOC_S%u_HEAP_%u", - numa_socket, heap->mz_count++); - - /* try getting a block. if we fail and we don't need as big a block - * as given in the config, we can shrink our request and try again - */ - do { - mz = rte_memzone_reserve(mz_name, mz_size, numa_socket, - mz_flags); - if (mz == NULL) - mz_size /= 2; - } while (mz == NULL && mz_size > min_size); - if (mz == NULL) - return -1; - - /* allocate the memory block headers, one at end, one at start */ - struct malloc_elem *start_elem = (struct malloc_elem *)mz->addr; - struct malloc_elem *end_elem = RTE_PTR_ADD(mz->addr, - mz_size - MALLOC_ELEM_OVERHEAD); - end_elem = RTE_PTR_ALIGN_FLOOR(end_elem, RTE_CACHE_LINE_SIZE); - - const unsigned elem_size = (uintptr_t)end_elem - (uintptr_t)start_elem; - malloc_elem_init(start_elem, heap, mz, elem_size); - malloc_elem_mkend(end_elem, start_elem); - malloc_elem_free_list_insert(start_elem); - - /* increase heap total size by size of new memzone */ - heap->total_size+=mz_size - MALLOC_ELEM_OVERHEAD; - return 0; -} - -/* - * Iterates through the freelist for a heap to find a free element - * which can store data of the required size and with the requested alignment. - * Returns null on failure, or pointer to element on success. - */ -static struct malloc_elem * -find_suitable_element(struct malloc_heap *heap, size_t size, unsigned align) -{ - size_t idx; - struct malloc_elem *elem; - - for (idx = malloc_elem_free_list_index(size); - idx < RTE_HEAP_NUM_FREELISTS; idx++) - { - for (elem = LIST_FIRST(&heap->free_head[idx]); - !!elem; elem = LIST_NEXT(elem, free_list)) - { - if (malloc_elem_can_hold(elem, size, align)) - return elem; - } - } - return NULL; -} - -/* - * Main function called by malloc to allocate a block of memory from the - * heap. It locks the free list, scans it, and adds a new memzone if the - * scan fails. Once the new memzone is added, it re-scans and should return - * the new element after releasing the lock. - */ -void * -malloc_heap_alloc(struct malloc_heap *heap, - const char *type __attribute__((unused)), size_t size, unsigned align) -{ - size = RTE_CACHE_LINE_ROUNDUP(size); - align = RTE_CACHE_LINE_ROUNDUP(align); - rte_spinlock_lock(&heap->lock); - struct malloc_elem *elem = find_suitable_element(heap, size, align); - if (elem == NULL){ - if ((malloc_heap_add_memzone(heap, size, align)) == 0) - elem = find_suitable_element(heap, size, align); - } - - if (elem != NULL){ - elem = malloc_elem_alloc(elem, size, align); - /* increase heap's count of allocated elements */ - heap->alloc_count++; - } - rte_spinlock_unlock(&heap->lock); - return elem == NULL ? NULL : (void *)(&elem[1]); - -} - -/* - * Function to retrieve data for heap on given socket - */ -int -malloc_heap_get_stats(const struct malloc_heap *heap, - struct rte_malloc_socket_stats *socket_stats) -{ - size_t idx; - struct malloc_elem *elem; - - /* Initialise variables for heap */ - socket_stats->free_count = 0; - socket_stats->heap_freesz_bytes = 0; - socket_stats->greatest_free_size = 0; - - /* Iterate through free list */ - for (idx = 0; idx < RTE_HEAP_NUM_FREELISTS; idx++) { - for (elem = LIST_FIRST(&heap->free_head[idx]); - !!elem; elem = LIST_NEXT(elem, free_list)) - { - socket_stats->free_count++; - socket_stats->heap_freesz_bytes += elem->size; - if (elem->size > socket_stats->greatest_free_size) - socket_stats->greatest_free_size = elem->size; - } - } - /* Get stats on overall heap and allocated memory on this heap */ - socket_stats->heap_totalsz_bytes = heap->total_size; - socket_stats->heap_allocsz_bytes = (socket_stats->heap_totalsz_bytes - - socket_stats->heap_freesz_bytes); - socket_stats->alloc_count = heap->alloc_count; - return 0; -} diff --git a/lib/librte_malloc/malloc_heap.h b/lib/librte_malloc/malloc_heap.h deleted file mode 100644 index a47136d..0000000 --- a/lib/librte_malloc/malloc_heap.h +++ /dev/null @@ -1,70 +0,0 @@ -/*- - * BSD LICENSE - * - * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in - * the documentation and/or other materials provided with the - * distribution. - * * Neither the name of Intel Corporation nor the names of its - * contributors may be used to endorse or promote products derived - * from this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - */ - -#ifndef MALLOC_HEAP_H_ -#define MALLOC_HEAP_H_ - -#include -#include - -#ifdef __cplusplus -extern "C" { -#endif - -static inline unsigned -malloc_get_numa_socket(void) -{ - unsigned socket_id = rte_socket_id(); - - if (socket_id == (unsigned)SOCKET_ID_ANY) - return 0; - - return socket_id; -} - -void * -malloc_heap_alloc(struct malloc_heap *heap, const char *type, - size_t size, unsigned align); - -int -malloc_heap_get_stats(const struct malloc_heap *heap, - struct rte_malloc_socket_stats *socket_stats); - -int -rte_eal_heap_memzone_init(void); - -#ifdef __cplusplus -} -#endif - -#endif /* MALLOC_HEAP_H_ */ diff --git a/lib/librte_malloc/rte_malloc.c b/lib/librte_malloc/rte_malloc.c index c313a57..4b9dc7f 100644 --- a/lib/librte_malloc/rte_malloc.c +++ b/lib/librte_malloc/rte_malloc.c @@ -31,230 +31,4 @@ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ -#include -#include -#include -#include -#include - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include -#include "malloc_elem.h" -#include "malloc_heap.h" - - -/* Free the memory space back to heap */ -void rte_free(void *addr) -{ - if (addr == NULL) return; - if (malloc_elem_free(malloc_elem_from_data(addr)) < 0) - rte_panic("Fatal error: Invalid memory\n"); -} - -/* - * Allocate memory on specified heap. - */ -void * -rte_malloc_socket(const char *type, size_t size, unsigned align, int socket_arg) -{ - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; - int socket, i; - void *ret; - - /* return NULL if size is 0 or alignment is not power-of-2 */ - if (size == 0 || (align && !rte_is_power_of_2(align))) - return NULL; - - if (socket_arg == SOCKET_ID_ANY) - socket = malloc_get_numa_socket(); - else - socket = socket_arg; - - /* Check socket parameter */ - if (socket >= RTE_MAX_NUMA_NODES) - return NULL; - - ret = malloc_heap_alloc(&mcfg->malloc_heaps[socket], type, - size, align == 0 ? 1 : align); - if (ret != NULL || socket_arg != SOCKET_ID_ANY) - return ret; - - /* try other heaps */ - for (i = 0; i < RTE_MAX_NUMA_NODES; i++) { - /* we already tried this one */ - if (i == socket) - continue; - - ret = malloc_heap_alloc(&mcfg->malloc_heaps[i], type, - size, align == 0 ? 1 : align); - if (ret != NULL) - return ret; - } - - return NULL; -} - -/* - * Allocate memory on default heap. - */ -void * -rte_malloc(const char *type, size_t size, unsigned align) -{ - return rte_malloc_socket(type, size, align, SOCKET_ID_ANY); -} - -/* - * Allocate zero'd memory on specified heap. - */ -void * -rte_zmalloc_socket(const char *type, size_t size, unsigned align, int socket) -{ - void *ptr = rte_malloc_socket(type, size, align, socket); - - if (ptr != NULL) - memset(ptr, 0, size); - return ptr; -} - -/* - * Allocate zero'd memory on default heap. - */ -void * -rte_zmalloc(const char *type, size_t size, unsigned align) -{ - return rte_zmalloc_socket(type, size, align, SOCKET_ID_ANY); -} - -/* - * Allocate zero'd memory on specified heap. - */ -void * -rte_calloc_socket(const char *type, size_t num, size_t size, unsigned align, int socket) -{ - return rte_zmalloc_socket(type, num * size, align, socket); -} - -/* - * Allocate zero'd memory on default heap. - */ -void * -rte_calloc(const char *type, size_t num, size_t size, unsigned align) -{ - return rte_zmalloc(type, num * size, align); -} - -/* - * Resize allocated memory. - */ -void * -rte_realloc(void *ptr, size_t size, unsigned align) -{ - if (ptr == NULL) - return rte_malloc(NULL, size, align); - - struct malloc_elem *elem = malloc_elem_from_data(ptr); - if (elem == NULL) - rte_panic("Fatal error: memory corruption detected\n"); - - size = RTE_CACHE_LINE_ROUNDUP(size), align = RTE_CACHE_LINE_ROUNDUP(align); - /* check alignment matches first, and if ok, see if we can resize block */ - if (RTE_PTR_ALIGN(ptr,align) == ptr && - malloc_elem_resize(elem, size) == 0) - return ptr; - - /* either alignment is off, or we have no room to expand, - * so move data. */ - void *new_ptr = rte_malloc(NULL, size, align); - if (new_ptr == NULL) - return NULL; - const unsigned old_size = elem->size - MALLOC_ELEM_OVERHEAD; - rte_memcpy(new_ptr, ptr, old_size < size ? old_size : size); - rte_free(ptr); - - return new_ptr; -} - -int -rte_malloc_validate(const void *ptr, size_t *size) -{ - const struct malloc_elem *elem = malloc_elem_from_data(ptr); - if (!malloc_elem_cookies_ok(elem)) - return -1; - if (size != NULL) - *size = elem->size - elem->pad - MALLOC_ELEM_OVERHEAD; - return 0; -} - -/* - * Function to retrieve data for heap on given socket - */ -int -rte_malloc_get_socket_stats(int socket, - struct rte_malloc_socket_stats *socket_stats) -{ - struct rte_mem_config *mcfg = rte_eal_get_configuration()->mem_config; - - if (socket >= RTE_MAX_NUMA_NODES || socket < 0) - return -1; - - return malloc_heap_get_stats(&mcfg->malloc_heaps[socket], socket_stats); -} - -/* - * Print stats on memory type. If type is NULL, info on all types is printed - */ -void -rte_malloc_dump_stats(FILE *f, __rte_unused const char *type) -{ - unsigned int socket; - struct rte_malloc_socket_stats sock_stats; - /* Iterate through all initialised heaps */ - for (socket=0; socket< RTE_MAX_NUMA_NODES; socket++) { - if ((rte_malloc_get_socket_stats(socket, &sock_stats) < 0)) - continue; - - fprintf(f, "Socket:%u\n", socket); - fprintf(f, "\tHeap_size:%zu,\n", sock_stats.heap_totalsz_bytes); - fprintf(f, "\tFree_size:%zu,\n", sock_stats.heap_freesz_bytes); - fprintf(f, "\tAlloc_size:%zu,\n", sock_stats.heap_allocsz_bytes); - fprintf(f, "\tGreatest_free_size:%zu,\n", - sock_stats.greatest_free_size); - fprintf(f, "\tAlloc_count:%u,\n",sock_stats.alloc_count); - fprintf(f, "\tFree_count:%u,\n", sock_stats.free_count); - } - return; -} - -/* - * TODO: Set limit to memory that can be allocated to memory type - */ -int -rte_malloc_set_limit(__rte_unused const char *type, - __rte_unused size_t max) -{ - return 0; -} - -/* - * Return the physical address of a virtual address obtained through rte_malloc - */ -phys_addr_t -rte_malloc_virt2phy(const void *addr) -{ - const struct malloc_elem *elem = malloc_elem_from_data(addr); - if (elem == NULL) - return 0; - return elem->mz->phys_addr + ((uintptr_t)addr - (uintptr_t)elem->mz->addr); -} +/* Empty file to be able to create a dummy library for deprecation policy */ diff --git a/lib/librte_malloc/rte_malloc.h b/lib/librte_malloc/rte_malloc.h deleted file mode 100644 index 74bb78c..0000000 --- a/lib/librte_malloc/rte_malloc.h +++ /dev/null @@ -1,342 +0,0 @@ -/*- - * BSD LICENSE - * - * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in - * the documentation and/or other materials provided with the - * distribution. - * * Neither the name of Intel Corporation nor the names of its - * contributors may be used to endorse or promote products derived - * from this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - */ - -#ifndef _RTE_MALLOC_H_ -#define _RTE_MALLOC_H_ - -/** - * @file - * RTE Malloc. This library provides methods for dynamically allocating memory - * from hugepages. - */ - -#include -#include -#include - -#ifdef __cplusplus -extern "C" { -#endif - -/** - * Structure to hold heap statistics obtained from rte_malloc_get_socket_stats function. - */ -struct rte_malloc_socket_stats { - size_t heap_totalsz_bytes; /**< Total bytes on heap */ - size_t heap_freesz_bytes; /**< Total free bytes on heap */ - size_t greatest_free_size; /**< Size in bytes of largest free block */ - unsigned free_count; /**< Number of free elements on heap */ - unsigned alloc_count; /**< Number of allocated elements on heap */ - size_t heap_allocsz_bytes; /**< Total allocated bytes on heap */ -}; - -/** - * This function allocates memory from the huge-page area of memory. The memory - * is not cleared. In NUMA systems, the memory allocated resides on the same - * NUMA socket as the core that calls this function. - * - * @param type - * A string identifying the type of allocated objects (useful for debug - * purposes, such as identifying the cause of a memory leak). Can be NULL. - * @param size - * Size (in bytes) to be allocated. - * @param align - * If 0, the return is a pointer that is suitably aligned for any kind of - * variable (in the same manner as malloc()). - * Otherwise, the return is a pointer that is a multiple of *align*. In - * this case, it must be a power of two. (Minimum alignment is the - * cacheline size, i.e. 64-bytes) - * @return - * - NULL on error. Not enough memory, or invalid arguments (size is 0, - * align is not a power of two). - * - Otherwise, the pointer to the allocated object. - */ -void * -rte_malloc(const char *type, size_t size, unsigned align); - -/** - * Allocate zero'ed memory from the heap. - * - * Equivalent to rte_malloc() except that the memory zone is - * initialised with zeros. In NUMA systems, the memory allocated resides on the - * same NUMA socket as the core that calls this function. - * - * @param type - * A string identifying the type of allocated objects (useful for debug - * purposes, such as identifying the cause of a memory leak). Can be NULL. - * @param size - * Size (in bytes) to be allocated. - * @param align - * If 0, the return is a pointer that is suitably aligned for any kind of - * variable (in the same manner as malloc()). - * Otherwise, the return is a pointer that is a multiple of *align*. In - * this case, it must obviously be a power of two. (Minimum alignment is the - * cacheline size, i.e. 64-bytes) - * @return - * - NULL on error. Not enough memory, or invalid arguments (size is 0, - * align is not a power of two). - * - Otherwise, the pointer to the allocated object. - */ -void * -rte_zmalloc(const char *type, size_t size, unsigned align); - -/** - * Replacement function for calloc(), using huge-page memory. Memory area is - * initialised with zeros. In NUMA systems, the memory allocated resides on the - * same NUMA socket as the core that calls this function. - * - * @param type - * A string identifying the type of allocated objects (useful for debug - * purposes, such as identifying the cause of a memory leak). Can be NULL. - * @param num - * Number of elements to be allocated. - * @param size - * Size (in bytes) of a single element. - * @param align - * If 0, the return is a pointer that is suitably aligned for any kind of - * variable (in the same manner as malloc()). - * Otherwise, the return is a pointer that is a multiple of *align*. In - * this case, it must obviously be a power of two. (Minimum alignment is the - * cacheline size, i.e. 64-bytes) - * @return - * - NULL on error. Not enough memory, or invalid arguments (size is 0, - * align is not a power of two). - * - Otherwise, the pointer to the allocated object. - */ -void * -rte_calloc(const char *type, size_t num, size_t size, unsigned align); - -/** - * Replacement function for realloc(), using huge-page memory. Reserved area - * memory is resized, preserving contents. In NUMA systems, the new area - * resides on the same NUMA socket as the old area. - * - * @param ptr - * Pointer to already allocated memory - * @param size - * Size (in bytes) of new area. If this is 0, memory is freed. - * @param align - * If 0, the return is a pointer that is suitably aligned for any kind of - * variable (in the same manner as malloc()). - * Otherwise, the return is a pointer that is a multiple of *align*. In - * this case, it must obviously be a power of two. (Minimum alignment is the - * cacheline size, i.e. 64-bytes) - * @return - * - NULL on error. Not enough memory, or invalid arguments (size is 0, - * align is not a power of two). - * - Otherwise, the pointer to the reallocated memory. - */ -void * -rte_realloc(void *ptr, size_t size, unsigned align); - -/** - * This function allocates memory from the huge-page area of memory. The memory - * is not cleared. - * - * @param type - * A string identifying the type of allocated objects (useful for debug - * purposes, such as identifying the cause of a memory leak). Can be NULL. - * @param size - * Size (in bytes) to be allocated. - * @param align - * If 0, the return is a pointer that is suitably aligned for any kind of - * variable (in the same manner as malloc()). - * Otherwise, the return is a pointer that is a multiple of *align*. In - * this case, it must be a power of two. (Minimum alignment is the - * cacheline size, i.e. 64-bytes) - * @param socket - * NUMA socket to allocate memory on. If SOCKET_ID_ANY is used, this function - * will behave the same as rte_malloc(). - * @return - * - NULL on error. Not enough memory, or invalid arguments (size is 0, - * align is not a power of two). - * - Otherwise, the pointer to the allocated object. - */ -void * -rte_malloc_socket(const char *type, size_t size, unsigned align, int socket); - -/** - * Allocate zero'ed memory from the heap. - * - * Equivalent to rte_malloc() except that the memory zone is - * initialised with zeros. - * - * @param type - * A string identifying the type of allocated objects (useful for debug - * purposes, such as identifying the cause of a memory leak). Can be NULL. - * @param size - * Size (in bytes) to be allocated. - * @param align - * If 0, the return is a pointer that is suitably aligned for any kind of - * variable (in the same manner as malloc()). - * Otherwise, the return is a pointer that is a multiple of *align*. In - * this case, it must obviously be a power of two. (Minimum alignment is the - * cacheline size, i.e. 64-bytes) - * @param socket - * NUMA socket to allocate memory on. If SOCKET_ID_ANY is used, this function - * will behave the same as rte_zmalloc(). - * @return - * - NULL on error. Not enough memory, or invalid arguments (size is 0, - * align is not a power of two). - * - Otherwise, the pointer to the allocated object. - */ -void * -rte_zmalloc_socket(const char *type, size_t size, unsigned align, int socket); - -/** - * Replacement function for calloc(), using huge-page memory. Memory area is - * initialised with zeros. - * - * @param type - * A string identifying the type of allocated objects (useful for debug - * purposes, such as identifying the cause of a memory leak). Can be NULL. - * @param num - * Number of elements to be allocated. - * @param size - * Size (in bytes) of a single element. - * @param align - * If 0, the return is a pointer that is suitably aligned for any kind of - * variable (in the same manner as malloc()). - * Otherwise, the return is a pointer that is a multiple of *align*. In - * this case, it must obviously be a power of two. (Minimum alignment is the - * cacheline size, i.e. 64-bytes) - * @param socket - * NUMA socket to allocate memory on. If SOCKET_ID_ANY is used, this function - * will behave the same as rte_calloc(). - * @return - * - NULL on error. Not enough memory, or invalid arguments (size is 0, - * align is not a power of two). - * - Otherwise, the pointer to the allocated object. - */ -void * -rte_calloc_socket(const char *type, size_t num, size_t size, unsigned align, int socket); - -/** - * Frees the memory space pointed to by the provided pointer. - * - * This pointer must have been returned by a previous call to - * rte_malloc(), rte_zmalloc(), rte_calloc() or rte_realloc(). The behaviour of - * rte_free() is undefined if the pointer does not match this requirement. - * - * If the pointer is NULL, the function does nothing. - * - * @param ptr - * The pointer to memory to be freed. - */ -void -rte_free(void *ptr); - -/** - * If malloc debug is enabled, check a memory block for header - * and trailer markers to indicate that all is well with the block. - * If size is non-null, also return the size of the block. - * - * @param ptr - * pointer to the start of a data block, must have been returned - * by a previous call to rte_malloc(), rte_zmalloc(), rte_calloc() - * or rte_realloc() - * @param size - * if non-null, and memory block pointer is valid, returns the size - * of the memory block - * @return - * -1 on error, invalid pointer passed or header and trailer markers - * are missing or corrupted - * 0 on success - */ -int -rte_malloc_validate(const void *ptr, size_t *size); - -/** - * Get heap statistics for the specified heap. - * - * @param socket - * An unsigned integer specifying the socket to get heap statistics for - * @param socket_stats - * A structure which provides memory to store statistics - * @return - * Null on error - * Pointer to structure storing statistics on success - */ -int -rte_malloc_get_socket_stats(int socket, - struct rte_malloc_socket_stats *socket_stats); - -/** - * Dump statistics. - * - * Dump for the specified type to the console. If the type argument is - * NULL, all memory types will be dumped. - * - * @param f - * A pointer to a file for output - * @param type - * A string identifying the type of objects to dump, or NULL - * to dump all objects. - */ -void -rte_malloc_dump_stats(FILE *f, const char *type); - -/** - * Set the maximum amount of allocated memory for this type. - * - * This is not yet implemented - * - * @param type - * A string identifying the type of allocated objects. - * @param max - * The maximum amount of allocated bytes for this type. - * @return - * - 0: Success. - * - (-1): Error. - */ -int -rte_malloc_set_limit(const char *type, size_t max); - -/** - * Return the physical address of a virtual address obtained through - * rte_malloc - * - * @param addr - * Adress obtained from a previous rte_malloc call - * @return - * NULL on error - * otherwise return physical address of the buffer - */ -phys_addr_t -rte_malloc_virt2phy(const void *addr); - -#ifdef __cplusplus -} -#endif - -#endif /* _RTE_MALLOC_H_ */ diff --git a/lib/librte_malloc/rte_malloc_version.map b/lib/librte_malloc/rte_malloc_version.map index af6ae9b..63cb5fc 100644 --- a/lib/librte_malloc/rte_malloc_version.map +++ b/lib/librte_malloc/rte_malloc_version.map @@ -1,19 +1,3 @@ DPDK_2.0 { - global: - - rte_calloc; - rte_calloc_socket; - rte_free; - rte_malloc; - rte_malloc_dump_stats; - rte_malloc_get_socket_stats; - rte_malloc_set_limit; - rte_malloc_socket; - rte_malloc_validate; - rte_malloc_virt2phy; - rte_realloc; - rte_zmalloc; - rte_zmalloc_socket; - local: *; }; diff --git a/lib/librte_mempool/Makefile b/lib/librte_mempool/Makefile index 940d1f7..a6898ef 100644 --- a/lib/librte_mempool/Makefile +++ b/lib/librte_mempool/Makefile @@ -48,8 +48,6 @@ endif # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_MEMPOOL)-include := rte_mempool.h -# this lib needs eal, rte_ring and rte_malloc DEPDIRS-$(CONFIG_RTE_LIBRTE_MEMPOOL) += lib/librte_eal lib/librte_ring -DEPDIRS-$(CONFIG_RTE_LIBRTE_MEMPOOL) += lib/librte_malloc include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_port/Makefile b/lib/librte_port/Makefile index de960fc..ddbb383 100644 --- a/lib/librte_port/Makefile +++ b/lib/librte_port/Makefile @@ -70,7 +70,6 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_PORT)-include += rte_port_source_sink.h DEPDIRS-$(CONFIG_RTE_LIBRTE_PORT) := lib/librte_eal DEPDIRS-$(CONFIG_RTE_LIBRTE_PORT) += lib/librte_mbuf DEPDIRS-$(CONFIG_RTE_LIBRTE_PORT) += lib/librte_mempool -DEPDIRS-$(CONFIG_RTE_LIBRTE_PORT) += lib/librte_malloc DEPDIRS-$(CONFIG_RTE_LIBRTE_PORT) += lib/librte_ether DEPDIRS-$(CONFIG_RTE_LIBRTE_PORT) += lib/librte_ip_frag diff --git a/lib/librte_ring/Makefile b/lib/librte_ring/Makefile index 84ad3d3..4b1112e 100644 --- a/lib/librte_ring/Makefile +++ b/lib/librte_ring/Makefile @@ -46,7 +46,6 @@ SRCS-$(CONFIG_RTE_LIBRTE_RING) := rte_ring.c # install includes SYMLINK-$(CONFIG_RTE_LIBRTE_RING)-include := rte_ring.h -# this lib needs eal and rte_malloc -DEPDIRS-$(CONFIG_RTE_LIBRTE_RING) += lib/librte_eal lib/librte_malloc +DEPDIRS-$(CONFIG_RTE_LIBRTE_RING) += lib/librte_eal include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_table/Makefile b/lib/librte_table/Makefile index 0d8394c..c5b3eaf 100644 --- a/lib/librte_table/Makefile +++ b/lib/librte_table/Makefile @@ -75,7 +75,6 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_stub.h DEPDIRS-$(CONFIG_RTE_LIBRTE_TABLE) := lib/librte_eal DEPDIRS-$(CONFIG_RTE_LIBRTE_TABLE) += lib/librte_mbuf DEPDIRS-$(CONFIG_RTE_LIBRTE_TABLE) += lib/librte_mempool -DEPDIRS-$(CONFIG_RTE_LIBRTE_TABLE) += lib/librte_malloc DEPDIRS-$(CONFIG_RTE_LIBRTE_TABLE) += lib/librte_port DEPDIRS-$(CONFIG_RTE_LIBRTE_TABLE) += lib/librte_lpm ifeq ($(CONFIG_RTE_LIBRTE_ACL),y) -- 1.9.3