From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8926A424CD; Tue, 11 Jun 2024 15:00:42 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4DF4540A7D; Tue, 11 Jun 2024 15:00:03 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mails.dpdk.org (Postfix) with ESMTP id 0387B4027D for ; Tue, 11 Jun 2024 14:59:57 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 18F0416F8; Tue, 11 Jun 2024 06:00:21 -0700 (PDT) Received: from ampere-altra-2-1.usa.Arm.com (ampere-altra-2-1.usa.arm.com [10.118.91.158]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 63CB63F5A1; Tue, 11 Jun 2024 05:59:56 -0700 (PDT) From: Paul Szczepanek To: dev@dpdk.org Cc: Paul Szczepanek , Honnappa Nagarahalli , Nathan Brown , Jack Bond-Preston Subject: [PATCH v15 5/6] docs: add pointer compression guide Date: Tue, 11 Jun 2024 12:59:46 +0000 Message-Id: <20240611125947.511243-6-paul.szczepanek@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240611125947.511243-1-paul.szczepanek@arm.com> References: <20230927150854.3670391-1-paul.szczepanek@arm.com> <20240611125947.511243-1-paul.szczepanek@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Documentation added in the prog guide for the new utility functions for pointer compression showing example code and potential usecases. Signed-off-by: Paul Szczepanek Reviewed-by: Honnappa Nagarahalli Reviewed-by: Nathan Brown Reviewed-by: Jack Bond-Preston --- MAINTAINERS | 1 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/ptr_compress_lib.rst | 160 +++++++++++++++++++++ doc/guides/rel_notes/release_24_07.rst | 2 + 4 files changed, 164 insertions(+) create mode 100644 doc/guides/prog_guide/ptr_compress_lib.rst diff --git a/MAINTAINERS b/MAINTAINERS index 27b2f03e6c..ed50121bd2 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1697,6 +1697,7 @@ F: lib/pci/ Pointer Compression M: Paul Szczepanek F: lib/ptr_compress/ +F: doc/guides/prog_guide/ptr_compress_lib.rst Power management M: Anatoly Burakov diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index d09d958e6c..6366849eb0 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -73,6 +73,7 @@ Programmer's Guide telemetry_lib bpf_lib graph_lib + ptr_compress_lib build-sdk-meson meson_ut build_app diff --git a/doc/guides/prog_guide/ptr_compress_lib.rst b/doc/guides/prog_guide/ptr_compress_lib.rst new file mode 100644 index 0000000000..5edbc35e56 --- /dev/null +++ b/doc/guides/prog_guide/ptr_compress_lib.rst @@ -0,0 +1,160 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2024 Arm Limited. + +Pointer Compression Library +=========================== + +Use ``rte_ptr_compress_16_shift()`` and ``rte_ptr_decompress_16_shift()`` to +compress and decompress pointers into 16-bit offsets. +Use ``rte_ptr_compress_32_shift()`` and ``rte_ptr_decompress_32_shift()`` to +compress and decompress pointers into 32-bit offsets. + +Compression takes advantage of the fact that pointers are usually located in a +limited memory region (like a mempool). By converting them to offsets from a +base memory address they can be stored in fewer bytes. How many bytes are needed +to store the offset is dictated by the memory region size and alignment of +objects the pointers point to. + +For example, a pointer which is part of a 4GB memory pool can be stored as 32 +bit offset. If the pointer points to memory that is 8 bytes aligned then 3 bits +can be dropped from the offset and a 32GB memory pool can now fit in 32 bits. + +For performance reasons these requirements are not enforced programmatically. +The programmer is responsible for ensuring that the combination of distance +from the base pointer and memory alignment allow for storing of the offset in +the number of bits indicated by the function name (16 or 32). Start of mempool +memory would be a good candidate for the base pointer. Otherwise any pointer +that precedes all pointers, is close enough and has the same alignment as the +pointers being compressed will work. + +Macros present in the rte_ptr_compress.h header may be used to evaluate whether +compression is possible: + +* RTE_PTR_COMPRESS_BITS_NEEDED_FOR_POINTER_WITHIN_RANGE + +* RTE_PTR_COMPRESS_BIT_SHIFT_FROM_ALIGNMENT + +* RTE_PTR_COMPRESS_CAN_COMPRESS_16_SHIFT + +* RTE_PTR_COMPRESS_CAN_COMPRESS_32_SHIFT + +These will help you calculate compression parameters and whether these are +legal for particular compression function. + +If using an rte_mempool you can get the parameters you need to use in the +compression macros and functions by using ``rte_mempool_get_mem_range()`` +and ``rte_mempool_get_obj_alignment()``. + +.. note:: + + Performance gains depend on the batch size of pointers and CPU capabilities + such as vector extensions. It's important to measure the performance + increase on target hardware. A test called ``ring_perf_autotest`` in + ``dpdk-test`` can provide the measurements. + +Example usage +~~~~~~~~~~~~~ + +In this example we send pointers between two cores through a ring. While this +is a realistic use case the code is simplified for demonstration purposes and +does not have error handling. + +.. code-block:: c + + #include + #include + #include + #include + + #define ITEMS_ARRAY_SIZE (1024) + #define BATCH_SIZE (128) + #define ALIGN_EXPONENT (3) + #define ITEM_ALIGN (1<