[dpdk-dev] [PATCH v3 20/20] doc: add external memory sample application guide

[Anatoly Burakov <anatoly.burakov@intel.com>]

Add a guide for external memory sample application. The application
is identical to Basic Forwarding example in everything except parts
of initialization code, so the bits that are identical will not be
described.

It is also not necessary to describe how external memory is being
allocated due to the expectation being that user will have their
own mechanisms to allocate memory outside of DPDK, and will only
be interested in how to integrate said memory into DPDK.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 doc/guides/sample_app_ug/external_mem.rst | 115 ++++++++++++++++++++++
 doc/guides/sample_app_ug/index.rst        |   1 +
 2 files changed, 116 insertions(+)
 create mode 100644 doc/guides/sample_app_ug/external_mem.rst

diff --git a/doc/guides/sample_app_ug/external_mem.rst b/doc/guides/sample_app_ug/external_mem.rst
new file mode 100644
index 000000000..594c3397a
--- /dev/null
+++ b/doc/guides/sample_app_ug/external_mem.rst
@@ -0,0 +1,115 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2015-2018 Intel Corporation.
+
+External Memory Sample Application
+==================================
+
+The External Memory sample application is a simple *skeleton* example of a
+forwarding application using externally allocated memory.
+
+It is intended as a demonstration of the basic workflow of using externally
+allocated memory in DPDK. This application is based on Basic Forwarding sample
+application, and differs only in its initialization path. For more detailed
+explanation of port initialization and packet forwarding code, please refer to
+*Basic Forwarding sample application user guide*.
+
+Compiling the Application
+-------------------------
+
+To compile the sample application see :doc:`compiling`.
+
+The application is located in the ``external_mem`` sub-directory.
+
+Running the Application
+-----------------------
+
+To run the example in a ``linuxapp`` environment:
+
+.. code-block:: console
+
+    ./build/extmem -l 1 -n 4
+
+Refer to *DPDK Getting Started Guide* for general information on running
+applications and the Environment Abstraction Layer (EAL) options.
+
+
+Explanation
+-----------
+
+For general overview of the code and explanation of the main components of this
+application, please refer to *Basic Forwarding sample application user guide*.
+This guide will only explain sections of the code relevant to using external
+memory in DPDK.
+
+All DPDK library functions used in the sample code are prefixed with ``rte_``
+and are explained in detail in the *DPDK API Documentation*.
+
+
+External Memory Initialization
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``main()`` function performs the initialization and calls the execution
+threads for each lcore.
+
+After initializing the Environment Abstraction Layer, the application also
+initializes external memory (in this case, it's allocating a chunk of memory
+using anonymous hugepages) inside the ``setup_extmem()`` local function.
+
+The first step in this process is to create an external heap:
+
+.. code-block:: c
+
+    ret = rte_malloc_heap_create(EXTMEM_HEAP_NAME);
+    if (ret < 0) {
+        printf("Cannot create heap\n");
+        return -1;
+    }
+
+Once the heap is created, ``create_extmem`` function is called to create the
+actual external memory area the application will be using. While the details of
+that process will not be described as they are not pertinent to the external
+memory API (it is expected that the user will have their own procedures to
+create external memory), there are a few important things to note.
+
+In order to add an externally allocated memory area to the newly created heap,
+the application needs the following pieces of information:
+
+* Pointer to start address of external memory area
+* Length of this area
+* Page size of memory backing this memory area
+* Optionally, a per-page IOVA table
+
+All of this information is to be provided by the user. Additionally, if VFIO is
+in use and if application intends to do DMA using the memory area, VFIO DMA
+mapping must also be performed using ``rte_vfio_dma_map`` function.
+
+Once the external memory is created and mapped for DMA, the application also has
+to add this memory to the heap that was created earlier:
+
+.. code-block:: c
+
+    ret = rte_malloc_heap_memory_add(EXTMEM_HEAP_NAME,
+            param.addr, param.len, param.iova_table,
+            param.iova_table_len, param.pgsz);
+
+If return value indicates success, the memory area has been successfully added
+to the heap. The next step is to retrieve the socket ID of this heap:
+
+.. code-block:: c
+
+    socket_id = rte_malloc_heap_get_socket(EXTMEM_HEAP_NAME);
+    if (socket_id < 0)
+        rte_exit(EXIT_FAILURE, "Invalid socket for external heap\n");
+
+After that, the socket ID has to be supplied to the mempool creation function:
+
+.. code-block:: c
+
+    mbuf_pool = rte_pktmbuf_pool_create("MBUF_POOL",
+        nb_mbufs_per_port * nb_ports, MBUF_CACHE_SIZE, 0,
+        mbuf_sz, socket_id);
+
+The rest of the application is identical to the Basic Forwarding example.
+
+The forwarding loop can be interrupted and the application closed using
+``Ctrl-C``.
diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
index 5bedf4f6f..0536edb8e 100644
--- a/doc/guides/sample_app_ug/index.rst
+++ b/doc/guides/sample_app_ug/index.rst
@@ -15,6 +15,7 @@ Sample Applications User Guides
     exception_path
     hello_world
     skeleton
+    external_mem
     rxtx_callbacks
     flow_classify
     flow_filtering
-- 
2.17.1