[dpdk-dev] [PATCH] doc: add Mellanox BlueField platform guide

[dpdk-dev] [PATCH] doc: add Mellanox BlueField platform guide

[Yongseok Koh]

Platform specific guide for Mellanox BlueField SoC is added.

Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
---
 .../linux_gsg/cross_build_dpdk_for_arm64.rst  |   2 +
 doc/guides/platform/bluefield.rst             | 141 ++++++++++++++++++
 doc/guides/platform/index.rst                 |   1 +
 doc/guides/rel_notes/release_19_05.rst        |   4 +
 4 files changed, 148 insertions(+)
 create mode 100644 doc/guides/platform/bluefield.rst

diff --git a/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst b/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
index f19e2edadb..4bf8f17ab0 100644
--- a/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
+++ b/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
@@ -34,6 +34,8 @@ Unzip and add into the PATH
 
    For the host requirements and other info, refer to the release note section: https://releases.linaro.org/components/toolchain/binaries/
 
+.. _getting_the_prerequisite_library:
+
 Getting the prerequisite library
 --------------------------------
 
diff --git a/doc/guides/platform/bluefield.rst b/doc/guides/platform/bluefield.rst
new file mode 100644
index 0000000000..c4ed306667
--- /dev/null
+++ b/doc/guides/platform/bluefield.rst
@@ -0,0 +1,141 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright 2019 Mellanox Technologies, Ltd
+
+Mellanox BlueField Board Support Package
+========================================
+
+This document has information about steps to setup Mellanox BlueField platform
+and common offload HW drivers of **Mellanox BlueField** family SoC.
+
+
+Supported BlueField family SoCs
+-------------------------------
+
+- `BlueField <http://www.mellanox.com/page/products_dyn?product_family=256&mtag=soc_overview>`_
+
+
+Supported BlueField Platforms
+-----------------------------
+
+- `BlueField SmartNIC <http://www.mellanox.com/page/products_dyn?product_family=275&mtag=bluefield_smart_nic>`_
+- `BlueField Reference Platforms <http://www.mellanox.com/page/products_dyn?product_family=286&mtag=bluefield_platforms>`_
+- `BlueField Controller Card <http://www.mellanox.com/page/products_dyn?product_family=288&mtag=bluefield_controller_card>`_
+
+
+Common Offload HW Drivers
+-------------------------
+
+1. **NIC Driver**
+
+   See :doc:`../nics/mlx5` for Mellanox mlx5 NIC driver information.
+
+2. **Cryptodev Driver**
+
+   This is based on the crypto extension support of armv8. See
+   :doc:`../cryptodevs/armv8` for armv8 crypto driver information.
+
+.. note::
+
+   BlueField has a variant having no armv8 crypto extension support.
+
+
+Steps To Setup Platform
+-----------------------
+
+Toolchains, OS and drivers can be downloaded and installed individually on the
+Web. But it is recommended to follow instructions at `Mellanox BlueField
+Software Website <http://www.mellanox.com/page/products_dyn?product_family=279&mtag=bluefield_software>`_.
+
+
+Compile DPDK
+------------
+
+DPDK can be compiled either natively on BlueField platforms or cross-compiled on
+an x86 based platform.
+
+Native Compilation
+~~~~~~~~~~~~~~~~~~
+
+Refer to :doc:`../nics/mlx5` for prerequisites. Either Mellanox OFED/EN or
+rdma-core library with corresponding kernel drivers is required.
+
+- make build
+
+.. code-block:: console
+
+        make config T=arm64-bluefield-linuxapp-gcc
+        make -j
+
+- meson build
+
+.. code-block:: console
+
+        meson build
+        ninja -C build
+
+Cross Compilation
+~~~~~~~~~~~~~~~~~
+
+Refer to :doc:`../linux_gsg/cross_build_dpdk_for_arm64` to install the cross
+toolchain for ARM64. Base on that, additional header files and libraries are
+required:
+
+   - libibverbs
+   - libmnl
+   - libmlx5
+   - libnl-3
+   - libnl-route-3
+
+Such required libraries can be cross-compiled and installed on to the cross
+toolchain directory like depicted in :ref:`getting_the_prerequisite_library`,
+but those can also be simply copied from the filesystem of a working BlueField
+platform. The following is to create such supplementary tarball on a BlueField
+platform.
+
+.. code-block:: console
+
+        mkdir -p aarch64-linux-gnu/libc
+        pushd $PWD
+        cd aarch64-linux-gnu/libc
+
+        # Copy libraries
+        mkdir -p lib64
+        cp -a /lib64/libibverbs* lib64/
+        cp -a /lib64/libmnl* lib64/
+        cp -a /lib64/libmlx5* lib64/
+        cp -a /lib64/libnl-3* lib64/
+        cp -a /lib64/libnl-route-3* lib64/
+
+        # Copy header files
+        mkdir -p usr/include/infiniband
+        mkdir -p usr/include/libmnl
+        cp -a /usr/include/infiniband/ib_user_ioctl_verbs.h usr/include/infiniband/
+        cp -a /usr/include/infiniband/mlx5*.h usr/include/infiniband/
+        cp -a /usr/include/infiniband/tm_types.h usr/include/infiniband/
+        cp -a /usr/include/infiniband/verbs*.h usr/include/infiniband/
+        cp -a /usr/include/libmnl/libmnl.h usr/include/libmnl/
+
+        # Create supplementary tarball
+        popd
+        tar cf aarch64-linux-gnu-mlx.tar aarch64-linux-gnu/
+
+Then, untar the tarball at the cross toolchain directory on the x86 host.
+
+.. code-block:: console
+
+        cd $(dirname $(which aarch64-linux-gnu-gcc))/..
+        tar xf aarch64-linux-gnu-mlx.tar
+
+- make build
+
+.. code-block:: console
+
+        make config T=arm64-bluefield-linuxapp-gcc
+        make -j CROSS=aarch64-linux-gnu- CONFIG_RTE_KNI_KMOD=n CONFIG_RTE_EAL_IGB_UIO=n
+
+- meson build
+
+.. code-block:: console
+
+        meson build --cross-file config/arm/arm64_bluefield_linux_gcc
+        ninja -C build
diff --git a/doc/guides/platform/index.rst b/doc/guides/platform/index.rst
index 98c66389d0..a17de2efba 100644
--- a/doc/guides/platform/index.rst
+++ b/doc/guides/platform/index.rst
@@ -10,6 +10,7 @@ The following are platform specific guides and setup information.
     :maxdepth: 2
     :numbered:
 
+    bluefield
     dpaa
     dpaa2
     octeontx
diff --git a/doc/guides/rel_notes/release_19_05.rst b/doc/guides/rel_notes/release_19_05.rst
index 5044ac7df1..5bd17c1bc5 100644
--- a/doc/guides/rel_notes/release_19_05.rst
+++ b/doc/guides/rel_notes/release_19_05.rst
@@ -60,6 +60,10 @@ New Features
   * OcteonTX2 (Marvell)
   * ThunderX2 (Marvell)
 
+* **Added new platform specific guide:**
+
+  * BlueField (Mellanox)
+
 * **Introduced Windows Support.**
 
   Added Windows support to build Hello World sample application.
-- 
2.21.0

[dpdk-dev] [PATCH] doc: add Mellanox BlueField platform guide

[Yongseok Koh]

Platform specific guide for Mellanox BlueField SoC is added.

Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
---
 .../linux_gsg/cross_build_dpdk_for_arm64.rst  |   2 +
 doc/guides/platform/bluefield.rst             | 141 ++++++++++++++++++
 doc/guides/platform/index.rst                 |   1 +
 doc/guides/rel_notes/release_19_05.rst        |   4 +
 4 files changed, 148 insertions(+)
 create mode 100644 doc/guides/platform/bluefield.rst

diff --git a/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst b/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
index f19e2edadb..4bf8f17ab0 100644
--- a/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
+++ b/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
@@ -34,6 +34,8 @@ Unzip and add into the PATH
 
    For the host requirements and other info, refer to the release note section: https://releases.linaro.org/components/toolchain/binaries/
 
+.. _getting_the_prerequisite_library:
+
 Getting the prerequisite library
 --------------------------------
 
diff --git a/doc/guides/platform/bluefield.rst b/doc/guides/platform/bluefield.rst
new file mode 100644
index 0000000000..c4ed306667
--- /dev/null
+++ b/doc/guides/platform/bluefield.rst
@@ -0,0 +1,141 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright 2019 Mellanox Technologies, Ltd
+
+Mellanox BlueField Board Support Package
+========================================
+
+This document has information about steps to setup Mellanox BlueField platform
+and common offload HW drivers of **Mellanox BlueField** family SoC.
+
+
+Supported BlueField family SoCs
+-------------------------------
+
+- `BlueField <http://www.mellanox.com/page/products_dyn?product_family=256&mtag=soc_overview>`_
+
+
+Supported BlueField Platforms
+-----------------------------
+
+- `BlueField SmartNIC <http://www.mellanox.com/page/products_dyn?product_family=275&mtag=bluefield_smart_nic>`_
+- `BlueField Reference Platforms <http://www.mellanox.com/page/products_dyn?product_family=286&mtag=bluefield_platforms>`_
+- `BlueField Controller Card <http://www.mellanox.com/page/products_dyn?product_family=288&mtag=bluefield_controller_card>`_
+
+
+Common Offload HW Drivers
+-------------------------
+
+1. **NIC Driver**
+
+   See :doc:`../nics/mlx5` for Mellanox mlx5 NIC driver information.
+
+2. **Cryptodev Driver**
+
+   This is based on the crypto extension support of armv8. See
+   :doc:`../cryptodevs/armv8` for armv8 crypto driver information.
+
+.. note::
+
+   BlueField has a variant having no armv8 crypto extension support.
+
+
+Steps To Setup Platform
+-----------------------
+
+Toolchains, OS and drivers can be downloaded and installed individually on the
+Web. But it is recommended to follow instructions at `Mellanox BlueField
+Software Website <http://www.mellanox.com/page/products_dyn?product_family=279&mtag=bluefield_software>`_.
+
+
+Compile DPDK
+------------
+
+DPDK can be compiled either natively on BlueField platforms or cross-compiled on
+an x86 based platform.
+
+Native Compilation
+~~~~~~~~~~~~~~~~~~
+
+Refer to :doc:`../nics/mlx5` for prerequisites. Either Mellanox OFED/EN or
+rdma-core library with corresponding kernel drivers is required.
+
+- make build
+
+.. code-block:: console
+
+        make config T=arm64-bluefield-linuxapp-gcc
+        make -j
+
+- meson build
+
+.. code-block:: console
+
+        meson build
+        ninja -C build
+
+Cross Compilation
+~~~~~~~~~~~~~~~~~
+
+Refer to :doc:`../linux_gsg/cross_build_dpdk_for_arm64` to install the cross
+toolchain for ARM64. Base on that, additional header files and libraries are
+required:
+
+   - libibverbs
+   - libmnl
+   - libmlx5
+   - libnl-3
+   - libnl-route-3
+
+Such required libraries can be cross-compiled and installed on to the cross
+toolchain directory like depicted in :ref:`getting_the_prerequisite_library`,
+but those can also be simply copied from the filesystem of a working BlueField
+platform. The following is to create such supplementary tarball on a BlueField
+platform.
+
+.. code-block:: console
+
+        mkdir -p aarch64-linux-gnu/libc
+        pushd $PWD
+        cd aarch64-linux-gnu/libc
+
+        # Copy libraries
+        mkdir -p lib64
+        cp -a /lib64/libibverbs* lib64/
+        cp -a /lib64/libmnl* lib64/
+        cp -a /lib64/libmlx5* lib64/
+        cp -a /lib64/libnl-3* lib64/
+        cp -a /lib64/libnl-route-3* lib64/
+
+        # Copy header files
+        mkdir -p usr/include/infiniband
+        mkdir -p usr/include/libmnl
+        cp -a /usr/include/infiniband/ib_user_ioctl_verbs.h usr/include/infiniband/
+        cp -a /usr/include/infiniband/mlx5*.h usr/include/infiniband/
+        cp -a /usr/include/infiniband/tm_types.h usr/include/infiniband/
+        cp -a /usr/include/infiniband/verbs*.h usr/include/infiniband/
+        cp -a /usr/include/libmnl/libmnl.h usr/include/libmnl/
+
+        # Create supplementary tarball
+        popd
+        tar cf aarch64-linux-gnu-mlx.tar aarch64-linux-gnu/
+
+Then, untar the tarball at the cross toolchain directory on the x86 host.
+
+.. code-block:: console
+
+        cd $(dirname $(which aarch64-linux-gnu-gcc))/..
+        tar xf aarch64-linux-gnu-mlx.tar
+
+- make build
+
+.. code-block:: console
+
+        make config T=arm64-bluefield-linuxapp-gcc
+        make -j CROSS=aarch64-linux-gnu- CONFIG_RTE_KNI_KMOD=n CONFIG_RTE_EAL_IGB_UIO=n
+
+- meson build
+
+.. code-block:: console
+
+        meson build --cross-file config/arm/arm64_bluefield_linux_gcc
+        ninja -C build
diff --git a/doc/guides/platform/index.rst b/doc/guides/platform/index.rst
index 98c66389d0..a17de2efba 100644
--- a/doc/guides/platform/index.rst
+++ b/doc/guides/platform/index.rst
@@ -10,6 +10,7 @@ The following are platform specific guides and setup information.
     :maxdepth: 2
     :numbered:
 
+    bluefield
     dpaa
     dpaa2
     octeontx
diff --git a/doc/guides/rel_notes/release_19_05.rst b/doc/guides/rel_notes/release_19_05.rst
index 5044ac7df1..5bd17c1bc5 100644
--- a/doc/guides/rel_notes/release_19_05.rst
+++ b/doc/guides/rel_notes/release_19_05.rst
@@ -60,6 +60,10 @@ New Features
   * OcteonTX2 (Marvell)
   * ThunderX2 (Marvell)
 
+* **Added new platform specific guide:**
+
+  * BlueField (Mellanox)
+
 * **Introduced Windows Support.**
 
   Added Windows support to build Hello World sample application.
-- 
2.21.0

Re: [dpdk-dev] [PATCH] doc: add Mellanox BlueField platform guide

[Thomas Monjalon]

08/05/2019 04:04, Yongseok Koh:
> --- a/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
> +++ b/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
> +.. _getting_the_prerequisite_library:

The name of this anchor should specify "arm".

> +
>  Getting the prerequisite library
>  --------------------------------
[...]
> --- /dev/null
> +++ b/doc/guides/platform/bluefield.rst
> +Toolchains, OS and drivers can be downloaded and installed individually on the
> +Web. But it is recommended to follow instructions at `Mellanox BlueField
> +Software Website <http://www.mellanox.com/page/products_dyn?product_family=279&mtag=bluefield_software>`_.

Please split lines more often to avoid a line break in the middle
of the link name.

[...]
> +Such required libraries can be cross-compiled and installed on to the cross
> +toolchain directory like depicted in :ref:`getting_the_prerequisite_library`,
> +but those can also be simply copied from the filesystem of a working BlueField
> +platform. The following is to create such supplementary tarball on a BlueField
> +platform.

We are talking about cross-compilation?
So we do the following commands on a x86 machine?
I think I don't understand, it seems to be on BlueField.
Please make it explicit below.

> +.. code-block:: console
> +
> +        mkdir -p aarch64-linux-gnu/libc
> +        pushd $PWD
> +        cd aarch64-linux-gnu/libc
> +
> +        # Copy libraries
> +        mkdir -p lib64
> +        cp -a /lib64/libibverbs* lib64/
> +        cp -a /lib64/libmnl* lib64/
> +        cp -a /lib64/libmlx5* lib64/
> +        cp -a /lib64/libnl-3* lib64/
> +        cp -a /lib64/libnl-route-3* lib64/
> +
> +        # Copy header files
> +        mkdir -p usr/include/infiniband
> +        mkdir -p usr/include/libmnl
> +        cp -a /usr/include/infiniband/ib_user_ioctl_verbs.h usr/include/infiniband/
> +        cp -a /usr/include/infiniband/mlx5*.h usr/include/infiniband/
> +        cp -a /usr/include/infiniband/tm_types.h usr/include/infiniband/
> +        cp -a /usr/include/infiniband/verbs*.h usr/include/infiniband/
> +        cp -a /usr/include/libmnl/libmnl.h usr/include/libmnl/
> +
> +        # Create supplementary tarball
> +        popd
> +        tar cf aarch64-linux-gnu-mlx.tar aarch64-linux-gnu/
> +
> +Then, untar the tarball at the cross toolchain directory on the x86 host.
> +
> +.. code-block:: console
> +
> +        cd $(dirname $(which aarch64-linux-gnu-gcc))/..
> +        tar xf aarch64-linux-gnu-mlx.tar

OK, this step is clearly on x86.

> +- make build

Should it be a sub-title instead of a list item?

> +
> +.. code-block:: console
> +
> +        make config T=arm64-bluefield-linuxapp-gcc
> +        make -j CROSS=aarch64-linux-gnu- CONFIG_RTE_KNI_KMOD=n CONFIG_RTE_EAL_IGB_UIO=n
> +
> +- meson build
> +
> +.. code-block:: console
> +
> +        meson build --cross-file config/arm/arm64_bluefield_linux_gcc
> +        ninja -C build

[...]
> --- a/doc/guides/rel_notes/release_19_05.rst
> +++ b/doc/guides/rel_notes/release_19_05.rst
> +* **Added new platform specific guide:**
> +
> +  * BlueField (Mellanox)

I don't think a doc deserves to be in the release notes.

Re: [dpdk-dev] [PATCH] doc: add Mellanox BlueField platform guide

[Thomas Monjalon]

08/05/2019 04:04, Yongseok Koh:
> --- a/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
> +++ b/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
> +.. _getting_the_prerequisite_library:

The name of this anchor should specify "arm".

> +
>  Getting the prerequisite library
>  --------------------------------
[...]
> --- /dev/null
> +++ b/doc/guides/platform/bluefield.rst
> +Toolchains, OS and drivers can be downloaded and installed individually on the
> +Web. But it is recommended to follow instructions at `Mellanox BlueField
> +Software Website <http://www.mellanox.com/page/products_dyn?product_family=279&mtag=bluefield_software>`_.

Please split lines more often to avoid a line break in the middle
of the link name.

[...]
> +Such required libraries can be cross-compiled and installed on to the cross
> +toolchain directory like depicted in :ref:`getting_the_prerequisite_library`,
> +but those can also be simply copied from the filesystem of a working BlueField
> +platform. The following is to create such supplementary tarball on a BlueField
> +platform.

We are talking about cross-compilation?
So we do the following commands on a x86 machine?
I think I don't understand, it seems to be on BlueField.
Please make it explicit below.

> +.. code-block:: console
> +
> +        mkdir -p aarch64-linux-gnu/libc
> +        pushd $PWD
> +        cd aarch64-linux-gnu/libc
> +
> +        # Copy libraries
> +        mkdir -p lib64
> +        cp -a /lib64/libibverbs* lib64/
> +        cp -a /lib64/libmnl* lib64/
> +        cp -a /lib64/libmlx5* lib64/
> +        cp -a /lib64/libnl-3* lib64/
> +        cp -a /lib64/libnl-route-3* lib64/
> +
> +        # Copy header files
> +        mkdir -p usr/include/infiniband
> +        mkdir -p usr/include/libmnl
> +        cp -a /usr/include/infiniband/ib_user_ioctl_verbs.h usr/include/infiniband/
> +        cp -a /usr/include/infiniband/mlx5*.h usr/include/infiniband/
> +        cp -a /usr/include/infiniband/tm_types.h usr/include/infiniband/
> +        cp -a /usr/include/infiniband/verbs*.h usr/include/infiniband/
> +        cp -a /usr/include/libmnl/libmnl.h usr/include/libmnl/
> +
> +        # Create supplementary tarball
> +        popd
> +        tar cf aarch64-linux-gnu-mlx.tar aarch64-linux-gnu/
> +
> +Then, untar the tarball at the cross toolchain directory on the x86 host.
> +
> +.. code-block:: console
> +
> +        cd $(dirname $(which aarch64-linux-gnu-gcc))/..
> +        tar xf aarch64-linux-gnu-mlx.tar

OK, this step is clearly on x86.

> +- make build

Should it be a sub-title instead of a list item?

> +
> +.. code-block:: console
> +
> +        make config T=arm64-bluefield-linuxapp-gcc
> +        make -j CROSS=aarch64-linux-gnu- CONFIG_RTE_KNI_KMOD=n CONFIG_RTE_EAL_IGB_UIO=n
> +
> +- meson build
> +
> +.. code-block:: console
> +
> +        meson build --cross-file config/arm/arm64_bluefield_linux_gcc
> +        ninja -C build

[...]
> --- a/doc/guides/rel_notes/release_19_05.rst
> +++ b/doc/guides/rel_notes/release_19_05.rst
> +* **Added new platform specific guide:**
> +
> +  * BlueField (Mellanox)

I don't think a doc deserves to be in the release notes.

Re: [dpdk-dev] [PATCH] doc: add Mellanox BlueField platform guide

[Yongseok Koh]

> On May 8, 2019, at 1:27 AM, Thomas Monjalon <thomas@monjalon.net> wrote:
> 
> 08/05/2019 04:04, Yongseok Koh:
>> --- a/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
>> +++ b/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
>> +.. _getting_the_prerequisite_library:
> 
> The name of this anchor should specify "arm".
> 
>> +
>> Getting the prerequisite library
>> --------------------------------
> [...]
>> --- /dev/null
>> +++ b/doc/guides/platform/bluefield.rst
>> +Toolchains, OS and drivers can be downloaded and installed individually on the
>> +Web. But it is recommended to follow instructions at `Mellanox BlueField
>> +Software Website <http://www.mellanox.com/page/products_dyn?product_family=279&mtag=bluefield_software>`_.
> 
> Please split lines more often to avoid a line break in the middle
> of the link name.
> 
> [...]
>> +Such required libraries can be cross-compiled and installed on to the cross
>> +toolchain directory like depicted in :ref:`getting_the_prerequisite_library`,
>> +but those can also be simply copied from the filesystem of a working BlueField
>> +platform. The following is to create such supplementary tarball on a BlueField
>> +platform.
> 
> We are talking about cross-compilation?
> So we do the following commands on a x86 machine?
> I think I don't understand, it seems to be on BlueField.
> Please make it explicit below.
> 
>> +.. code-block:: console
>> +
>> +        mkdir -p aarch64-linux-gnu/libc
>> +        pushd $PWD
>> +        cd aarch64-linux-gnu/libc
>> +
>> +        # Copy libraries
>> +        mkdir -p lib64
>> +        cp -a /lib64/libibverbs* lib64/
>> +        cp -a /lib64/libmnl* lib64/
>> +        cp -a /lib64/libmlx5* lib64/
>> +        cp -a /lib64/libnl-3* lib64/
>> +        cp -a /lib64/libnl-route-3* lib64/
>> +
>> +        # Copy header files
>> +        mkdir -p usr/include/infiniband
>> +        mkdir -p usr/include/libmnl
>> +        cp -a /usr/include/infiniband/ib_user_ioctl_verbs.h usr/include/infiniband/
>> +        cp -a /usr/include/infiniband/mlx5*.h usr/include/infiniband/
>> +        cp -a /usr/include/infiniband/tm_types.h usr/include/infiniband/
>> +        cp -a /usr/include/infiniband/verbs*.h usr/include/infiniband/
>> +        cp -a /usr/include/libmnl/libmnl.h usr/include/libmnl/
>> +
>> +        # Create supplementary tarball
>> +        popd
>> +        tar cf aarch64-linux-gnu-mlx.tar aarch64-linux-gnu/
>> +
>> +Then, untar the tarball at the cross toolchain directory on the x86 host.
>> +
>> +.. code-block:: console
>> +
>> +        cd $(dirname $(which aarch64-linux-gnu-gcc))/..
>> +        tar xf aarch64-linux-gnu-mlx.tar
> 
> OK, this step is clearly on x86.
> 
>> +- make build
> 
> Should it be a sub-title instead of a list item?
> 
>> +
>> +.. code-block:: console
>> +
>> +        make config T=arm64-bluefield-linuxapp-gcc
>> +        make -j CROSS=aarch64-linux-gnu- CONFIG_RTE_KNI_KMOD=n CONFIG_RTE_EAL_IGB_UIO=n
>> +
>> +- meson build
>> +
>> +.. code-block:: console
>> +
>> +        meson build --cross-file config/arm/arm64_bluefield_linux_gcc
>> +        ninja -C build
> 
> [...]
>> --- a/doc/guides/rel_notes/release_19_05.rst
>> +++ b/doc/guides/rel_notes/release_19_05.rst
>> +* **Added new platform specific guide:**
>> +
>> +  * BlueField (Mellanox)
> 
> I don't think a doc deserves to be in the release notes.

Thanks for the review, Thomas.
Will fix what you pointed out, all of them make sense to me.
I'm going to wait for another review (hopefully) and send out v2 soon.

Yongseok

Re: [dpdk-dev] [PATCH] doc: add Mellanox BlueField platform guide

[Yongseok Koh]

> On May 8, 2019, at 1:27 AM, Thomas Monjalon <thomas@monjalon.net> wrote:
> 
> 08/05/2019 04:04, Yongseok Koh:
>> --- a/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
>> +++ b/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
>> +.. _getting_the_prerequisite_library:
> 
> The name of this anchor should specify "arm".
> 
>> +
>> Getting the prerequisite library
>> --------------------------------
> [...]
>> --- /dev/null
>> +++ b/doc/guides/platform/bluefield.rst
>> +Toolchains, OS and drivers can be downloaded and installed individually on the
>> +Web. But it is recommended to follow instructions at `Mellanox BlueField
>> +Software Website <http://www.mellanox.com/page/products_dyn?product_family=279&mtag=bluefield_software>`_.
> 
> Please split lines more often to avoid a line break in the middle
> of the link name.
> 
> [...]
>> +Such required libraries can be cross-compiled and installed on to the cross
>> +toolchain directory like depicted in :ref:`getting_the_prerequisite_library`,
>> +but those can also be simply copied from the filesystem of a working BlueField
>> +platform. The following is to create such supplementary tarball on a BlueField
>> +platform.
> 
> We are talking about cross-compilation?
> So we do the following commands on a x86 machine?
> I think I don't understand, it seems to be on BlueField.
> Please make it explicit below.
> 
>> +.. code-block:: console
>> +
>> +        mkdir -p aarch64-linux-gnu/libc
>> +        pushd $PWD
>> +        cd aarch64-linux-gnu/libc
>> +
>> +        # Copy libraries
>> +        mkdir -p lib64
>> +        cp -a /lib64/libibverbs* lib64/
>> +        cp -a /lib64/libmnl* lib64/
>> +        cp -a /lib64/libmlx5* lib64/
>> +        cp -a /lib64/libnl-3* lib64/
>> +        cp -a /lib64/libnl-route-3* lib64/
>> +
>> +        # Copy header files
>> +        mkdir -p usr/include/infiniband
>> +        mkdir -p usr/include/libmnl
>> +        cp -a /usr/include/infiniband/ib_user_ioctl_verbs.h usr/include/infiniband/
>> +        cp -a /usr/include/infiniband/mlx5*.h usr/include/infiniband/
>> +        cp -a /usr/include/infiniband/tm_types.h usr/include/infiniband/
>> +        cp -a /usr/include/infiniband/verbs*.h usr/include/infiniband/
>> +        cp -a /usr/include/libmnl/libmnl.h usr/include/libmnl/
>> +
>> +        # Create supplementary tarball
>> +        popd
>> +        tar cf aarch64-linux-gnu-mlx.tar aarch64-linux-gnu/
>> +
>> +Then, untar the tarball at the cross toolchain directory on the x86 host.
>> +
>> +.. code-block:: console
>> +
>> +        cd $(dirname $(which aarch64-linux-gnu-gcc))/..
>> +        tar xf aarch64-linux-gnu-mlx.tar
> 
> OK, this step is clearly on x86.
> 
>> +- make build
> 
> Should it be a sub-title instead of a list item?
> 
>> +
>> +.. code-block:: console
>> +
>> +        make config T=arm64-bluefield-linuxapp-gcc
>> +        make -j CROSS=aarch64-linux-gnu- CONFIG_RTE_KNI_KMOD=n CONFIG_RTE_EAL_IGB_UIO=n
>> +
>> +- meson build
>> +
>> +.. code-block:: console
>> +
>> +        meson build --cross-file config/arm/arm64_bluefield_linux_gcc
>> +        ninja -C build
> 
> [...]
>> --- a/doc/guides/rel_notes/release_19_05.rst
>> +++ b/doc/guides/rel_notes/release_19_05.rst
>> +* **Added new platform specific guide:**
>> +
>> +  * BlueField (Mellanox)
> 
> I don't think a doc deserves to be in the release notes.

Thanks for the review, Thomas.
Will fix what you pointed out, all of them make sense to me.
I'm going to wait for another review (hopefully) and send out v2 soon.

Yongseok

[dpdk-dev] [PATCH v2] doc: add Mellanox BlueField platform guide

[Yongseok Koh]

Platform specific guide for Mellanox BlueField SoC is added.

Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
---

v2:
* remove release note update
* cosmetic changes
* make contents clearer

 .../linux_gsg/cross_build_dpdk_for_arm64.rst  |   2 +
 doc/guides/platform/bluefield.rst             | 147 ++++++++++++++++++
 doc/guides/platform/index.rst                 |   1 +
 3 files changed, 150 insertions(+)
 create mode 100644 doc/guides/platform/bluefield.rst

diff --git a/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst b/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
index f19e2edadb..bba9b92039 100644
--- a/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
+++ b/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
@@ -34,6 +34,8 @@ Unzip and add into the PATH
 
    For the host requirements and other info, refer to the release note section: https://releases.linaro.org/components/toolchain/binaries/
 
+.. _arm_cross_build_getting_the_prerequisite_library:
+
 Getting the prerequisite library
 --------------------------------
 
diff --git a/doc/guides/platform/bluefield.rst b/doc/guides/platform/bluefield.rst
new file mode 100644
index 0000000000..0bb58e50ac
--- /dev/null
+++ b/doc/guides/platform/bluefield.rst
@@ -0,0 +1,147 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright 2019 Mellanox Technologies, Ltd
+
+Mellanox BlueField Board Support Package
+========================================
+
+This document has information about steps to setup Mellanox BlueField platform
+and common offload HW drivers of **Mellanox BlueField** family SoC.
+
+
+Supported BlueField family SoCs
+-------------------------------
+
+- `BlueField <http://www.mellanox.com/page/products_dyn?product_family=256&mtag=soc_overview>`_
+
+
+Supported BlueField Platforms
+-----------------------------
+
+- `BlueField SmartNIC <http://www.mellanox.com/page/products_dyn?product_family=275&mtag=bluefield_smart_nic>`_
+- `BlueField Reference Platforms <http://www.mellanox.com/page/products_dyn?product_family=286&mtag=bluefield_platforms>`_
+- `BlueField Controller Card <http://www.mellanox.com/page/products_dyn?product_family=288&mtag=bluefield_controller_card>`_
+
+
+Common Offload HW Drivers
+-------------------------
+
+1. **NIC Driver**
+
+   See :doc:`../nics/mlx5` for Mellanox mlx5 NIC driver information.
+
+2. **Cryptodev Driver**
+
+   This is based on the crypto extension support of armv8. See
+   :doc:`../cryptodevs/armv8` for armv8 crypto driver information.
+
+.. note::
+
+   BlueField has a variant having no armv8 crypto extension support.
+
+
+Steps To Setup Platform
+-----------------------
+
+Toolchains, OS and drivers can be downloaded and installed individually from the
+Web. But it is recommended to follow instructions at
+`Mellanox BlueField Software Website
+<http://www.mellanox.com/page/products_dyn?product_family=279&mtag=bluefield_software>`_.
+
+
+Compile DPDK
+------------
+
+DPDK can be compiled either natively on BlueField platforms or cross-compiled on
+an x86 based platform.
+
+Native Compilation
+~~~~~~~~~~~~~~~~~~
+
+Refer to :doc:`../nics/mlx5` for prerequisites. Either Mellanox OFED/EN or
+rdma-core library with corresponding kernel drivers is required.
+
+make build
+^^^^^^^^^^
+
+.. code-block:: console
+
+        make config T=arm64-bluefield-linuxapp-gcc
+        make -j
+
+meson build
+^^^^^^^^^^^
+
+.. code-block:: console
+
+        meson build
+        ninja -C build
+
+Cross Compilation
+~~~~~~~~~~~~~~~~~
+
+Refer to :doc:`../linux_gsg/cross_build_dpdk_for_arm64` to install the cross
+toolchain for ARM64. Base on that, additional header files and libraries are
+required:
+
+   - libibverbs
+   - libmnl
+   - libmlx5
+   - libnl-3
+   - libnl-route-3
+
+Such header files and libraries can be cross-compiled and installed on to the
+cross toolchain directory like depicted in
+:ref:`arm_cross_build_getting_the_prerequisite_library`, but those can also be
+simply copied from the filesystem of a working BlueField platform. The following
+script can be run on a BlueField platform in order to create a supplementary
+tarball for the cross toolchain.
+
+.. code-block:: console
+
+        mkdir -p aarch64-linux-gnu/libc
+        pushd $PWD
+        cd aarch64-linux-gnu/libc
+
+        # Copy libraries
+        mkdir -p lib64
+        cp -a /lib64/libibverbs* lib64/
+        cp -a /lib64/libmnl* lib64/
+        cp -a /lib64/libmlx5* lib64/
+        cp -a /lib64/libnl-3* lib64/
+        cp -a /lib64/libnl-route-3* lib64/
+
+        # Copy header files
+        mkdir -p usr/include/infiniband
+        mkdir -p usr/include/libmnl
+        cp -a /usr/include/infiniband/ib_user_ioctl_verbs.h usr/include/infiniband/
+        cp -a /usr/include/infiniband/mlx5*.h usr/include/infiniband/
+        cp -a /usr/include/infiniband/tm_types.h usr/include/infiniband/
+        cp -a /usr/include/infiniband/verbs*.h usr/include/infiniband/
+        cp -a /usr/include/libmnl/libmnl.h usr/include/libmnl/
+
+        # Create supplementary tarball
+        popd
+        tar cf aarch64-linux-gnu-mlx.tar aarch64-linux-gnu/
+
+Then, untar the tarball at the cross toolchain directory on the x86 host.
+
+.. code-block:: console
+
+        cd $(dirname $(which aarch64-linux-gnu-gcc))/..
+        tar xf aarch64-linux-gnu-mlx.tar
+
+make build
+^^^^^^^^^^
+
+.. code-block:: console
+
+        make config T=arm64-bluefield-linuxapp-gcc
+        make -j CROSS=aarch64-linux-gnu- CONFIG_RTE_KNI_KMOD=n CONFIG_RTE_EAL_IGB_UIO=n
+
+meson build
+^^^^^^^^^^^
+
+.. code-block:: console
+
+        meson build --cross-file config/arm/arm64_bluefield_linux_gcc
+        ninja -C build
diff --git a/doc/guides/platform/index.rst b/doc/guides/platform/index.rst
index 98c66389d0..a17de2efba 100644
--- a/doc/guides/platform/index.rst
+++ b/doc/guides/platform/index.rst
@@ -10,6 +10,7 @@ The following are platform specific guides and setup information.
     :maxdepth: 2
     :numbered:
 
+    bluefield
     dpaa
     dpaa2
     octeontx
-- 
2.21.0

[dpdk-dev] [PATCH v2] doc: add Mellanox BlueField platform guide

[Yongseok Koh]

Platform specific guide for Mellanox BlueField SoC is added.

Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
---

v2:
* remove release note update
* cosmetic changes
* make contents clearer

 .../linux_gsg/cross_build_dpdk_for_arm64.rst  |   2 +
 doc/guides/platform/bluefield.rst             | 147 ++++++++++++++++++
 doc/guides/platform/index.rst                 |   1 +
 3 files changed, 150 insertions(+)
 create mode 100644 doc/guides/platform/bluefield.rst

diff --git a/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst b/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
index f19e2edadb..bba9b92039 100644
--- a/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
+++ b/doc/guides/linux_gsg/cross_build_dpdk_for_arm64.rst
@@ -34,6 +34,8 @@ Unzip and add into the PATH
 
    For the host requirements and other info, refer to the release note section: https://releases.linaro.org/components/toolchain/binaries/
 
+.. _arm_cross_build_getting_the_prerequisite_library:
+
 Getting the prerequisite library
 --------------------------------
 
diff --git a/doc/guides/platform/bluefield.rst b/doc/guides/platform/bluefield.rst
new file mode 100644
index 0000000000..0bb58e50ac
--- /dev/null
+++ b/doc/guides/platform/bluefield.rst
@@ -0,0 +1,147 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright 2019 Mellanox Technologies, Ltd
+
+Mellanox BlueField Board Support Package
+========================================
+
+This document has information about steps to setup Mellanox BlueField platform
+and common offload HW drivers of **Mellanox BlueField** family SoC.
+
+
+Supported BlueField family SoCs
+-------------------------------
+
+- `BlueField <http://www.mellanox.com/page/products_dyn?product_family=256&mtag=soc_overview>`_
+
+
+Supported BlueField Platforms
+-----------------------------
+
+- `BlueField SmartNIC <http://www.mellanox.com/page/products_dyn?product_family=275&mtag=bluefield_smart_nic>`_
+- `BlueField Reference Platforms <http://www.mellanox.com/page/products_dyn?product_family=286&mtag=bluefield_platforms>`_
+- `BlueField Controller Card <http://www.mellanox.com/page/products_dyn?product_family=288&mtag=bluefield_controller_card>`_
+
+
+Common Offload HW Drivers
+-------------------------
+
+1. **NIC Driver**
+
+   See :doc:`../nics/mlx5` for Mellanox mlx5 NIC driver information.
+
+2. **Cryptodev Driver**
+
+   This is based on the crypto extension support of armv8. See
+   :doc:`../cryptodevs/armv8` for armv8 crypto driver information.
+
+.. note::
+
+   BlueField has a variant having no armv8 crypto extension support.
+
+
+Steps To Setup Platform
+-----------------------
+
+Toolchains, OS and drivers can be downloaded and installed individually from the
+Web. But it is recommended to follow instructions at
+`Mellanox BlueField Software Website
+<http://www.mellanox.com/page/products_dyn?product_family=279&mtag=bluefield_software>`_.
+
+
+Compile DPDK
+------------
+
+DPDK can be compiled either natively on BlueField platforms or cross-compiled on
+an x86 based platform.
+
+Native Compilation
+~~~~~~~~~~~~~~~~~~
+
+Refer to :doc:`../nics/mlx5` for prerequisites. Either Mellanox OFED/EN or
+rdma-core library with corresponding kernel drivers is required.
+
+make build
+^^^^^^^^^^
+
+.. code-block:: console
+
+        make config T=arm64-bluefield-linuxapp-gcc
+        make -j
+
+meson build
+^^^^^^^^^^^
+
+.. code-block:: console
+
+        meson build
+        ninja -C build
+
+Cross Compilation
+~~~~~~~~~~~~~~~~~
+
+Refer to :doc:`../linux_gsg/cross_build_dpdk_for_arm64` to install the cross
+toolchain for ARM64. Base on that, additional header files and libraries are
+required:
+
+   - libibverbs
+   - libmnl
+   - libmlx5
+   - libnl-3
+   - libnl-route-3
+
+Such header files and libraries can be cross-compiled and installed on to the
+cross toolchain directory like depicted in
+:ref:`arm_cross_build_getting_the_prerequisite_library`, but those can also be
+simply copied from the filesystem of a working BlueField platform. The following
+script can be run on a BlueField platform in order to create a supplementary
+tarball for the cross toolchain.
+
+.. code-block:: console
+
+        mkdir -p aarch64-linux-gnu/libc
+        pushd $PWD
+        cd aarch64-linux-gnu/libc
+
+        # Copy libraries
+        mkdir -p lib64
+        cp -a /lib64/libibverbs* lib64/
+        cp -a /lib64/libmnl* lib64/
+        cp -a /lib64/libmlx5* lib64/
+        cp -a /lib64/libnl-3* lib64/
+        cp -a /lib64/libnl-route-3* lib64/
+
+        # Copy header files
+        mkdir -p usr/include/infiniband
+        mkdir -p usr/include/libmnl
+        cp -a /usr/include/infiniband/ib_user_ioctl_verbs.h usr/include/infiniband/
+        cp -a /usr/include/infiniband/mlx5*.h usr/include/infiniband/
+        cp -a /usr/include/infiniband/tm_types.h usr/include/infiniband/
+        cp -a /usr/include/infiniband/verbs*.h usr/include/infiniband/
+        cp -a /usr/include/libmnl/libmnl.h usr/include/libmnl/
+
+        # Create supplementary tarball
+        popd
+        tar cf aarch64-linux-gnu-mlx.tar aarch64-linux-gnu/
+
+Then, untar the tarball at the cross toolchain directory on the x86 host.
+
+.. code-block:: console
+
+        cd $(dirname $(which aarch64-linux-gnu-gcc))/..
+        tar xf aarch64-linux-gnu-mlx.tar
+
+make build
+^^^^^^^^^^
+
+.. code-block:: console
+
+        make config T=arm64-bluefield-linuxapp-gcc
+        make -j CROSS=aarch64-linux-gnu- CONFIG_RTE_KNI_KMOD=n CONFIG_RTE_EAL_IGB_UIO=n
+
+meson build
+^^^^^^^^^^^
+
+.. code-block:: console
+
+        meson build --cross-file config/arm/arm64_bluefield_linux_gcc
+        ninja -C build
diff --git a/doc/guides/platform/index.rst b/doc/guides/platform/index.rst
index 98c66389d0..a17de2efba 100644
--- a/doc/guides/platform/index.rst
+++ b/doc/guides/platform/index.rst
@@ -10,6 +10,7 @@ The following are platform specific guides and setup information.
     :maxdepth: 2
     :numbered:
 
+    bluefield
     dpaa
     dpaa2
     octeontx
-- 
2.21.0

Re: [dpdk-dev] [PATCH v2] doc: add Mellanox BlueField platform guide

[Mcnamara, John]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yongseok Koh
> Sent: Thursday, May 9, 2019 12:39 AM
> To: thomas@monjalon.net
> Cc: dev@dpdk.org; jerinj@marvell.com; shahafs@mellanox.com;
> Honnappa.Nagarahalli@arm.com
> Subject: [dpdk-dev] [PATCH v2] doc: add Mellanox BlueField platform guide
> 
> Platform specific guide for Mellanox BlueField SoC is added.
> 
> Signed-off-by: Yongseok Koh <yskoh@mellanox.com>

Acked-by: John McNamara <john.mcnamara@intel.com>

Re: [dpdk-dev] [PATCH v2] doc: add Mellanox BlueField platform guide

[Mcnamara, John]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yongseok Koh
> Sent: Thursday, May 9, 2019 12:39 AM
> To: thomas@monjalon.net
> Cc: dev@dpdk.org; jerinj@marvell.com; shahafs@mellanox.com;
> Honnappa.Nagarahalli@arm.com
> Subject: [dpdk-dev] [PATCH v2] doc: add Mellanox BlueField platform guide
> 
> Platform specific guide for Mellanox BlueField SoC is added.
> 
> Signed-off-by: Yongseok Koh <yskoh@mellanox.com>

Acked-by: John McNamara <john.mcnamara@intel.com>

Re: [dpdk-dev] [PATCH v2] doc: add Mellanox BlueField platform guide

[Thomas Monjalon]

> > Platform specific guide for Mellanox BlueField SoC is added.
> > 
> > Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
> 
> Acked-by: John McNamara <john.mcnamara@intel.com>

Applied, thanks

Re: [dpdk-dev] [PATCH v2] doc: add Mellanox BlueField platform guide

[Thomas Monjalon]

> > Platform specific guide for Mellanox BlueField SoC is added.
> > 
> > Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
> 
> Acked-by: John McNamara <john.mcnamara@intel.com>

Applied, thanks