[RFC v2 0/3] Import Kernel uAPI header files

[RFC v2 0/3] Import Kernel uAPI header files

[Maxime Coquelin]

This series enables importing Linux Kernel uAPI headers
into the DPDK repository. It aims at solving alignment
issues between the build system and the system applications
linked ot DPDK libraries are run on.

It can also help simplify spaghetti code done to support
different versions of the Linux Kernel headers used by
the build system.

Guidelines and import script are also part of first patch.
A uAPI checker script is added in this 2nd RFC, goals is to
use it in CI for any patch touching linux-headers/uapi.

Follow-up patches are an example of imported uAPI inclusion
of VDUSE header into the Vhost library.

Morten, I did not apply your Ack on patch 1, as it has
some significant changes and additions.

Changes in RFC v2:
==================
- Fix typos in documentation and commit messags (David, Morten)
- Add uAPI checker script
- Add uAPI to global_inc
- Fix build issues on FreeBSD and documentation (CI, David)
- Simplify import script (David)

Maxime Coquelin (3):
  uapi: introduce kernel uAPI headers import
  uapi: import VDUSE header
  vduse: use import VDUSE uAPI header

 devtools/check-linux-uapi.sh           |  74 ++++++
 devtools/import-linux-uapi.sh          |  45 ++++
 doc/guides/contributing/index.rst      |   1 +
 doc/guides/contributing/linux_uapi.rst |  77 ++++++
 lib/vhost/meson.build                  |   5 +-
 lib/vhost/vduse.c                      |   2 +-
 lib/vhost/vduse.h                      |  22 --
 linux-headers/uapi/.gitignore          |   3 +
 linux-headers/uapi/linux/vduse.h       | 353 +++++++++++++++++++++++++
 meson.build                            |   8 +-
 10 files changed, 561 insertions(+), 29 deletions(-)
 create mode 100755 devtools/check-linux-uapi.sh
 create mode 100755 devtools/import-linux-uapi.sh
 create mode 100644 doc/guides/contributing/linux_uapi.rst
 create mode 100644 linux-headers/uapi/.gitignore
 create mode 100644 linux-headers/uapi/linux/vduse.h

-- 
2.46.0

[RFC v2 1/3] uapi: introduce kernel uAPI headers import

[Maxime Coquelin]

This patch introduces uAPI headers import into the DPDK
repository. This import is possible thanks to Linux Kernel
licence exception for syscalls:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/LICENSES/exceptions/Linux-syscall-note

Header files are have to be explicitly imported, and
libraries and drivers have to explicitly enable their
inclusion.

Guidelines are provided in the documentation, and helper
scripts are also provided to ensure proper importation of the
header (unmodified content from a released Kernel version):
 - import-linux-uapi.sh: used to download and install a given
header to linux-headers/uapi/
 - check-linux-uapi.sh: used to check all headers are valid

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
 devtools/check-linux-uapi.sh           | 74 +++++++++++++++++++++++++
 devtools/import-linux-uapi.sh          | 45 +++++++++++++++
 doc/guides/contributing/index.rst      |  1 +
 doc/guides/contributing/linux_uapi.rst | 77 ++++++++++++++++++++++++++
 linux-headers/uapi/.gitignore          |  3 +
 meson.build                            |  8 ++-
 6 files changed, 206 insertions(+), 2 deletions(-)
 create mode 100755 devtools/check-linux-uapi.sh
 create mode 100755 devtools/import-linux-uapi.sh
 create mode 100644 doc/guides/contributing/linux_uapi.rst
 create mode 100644 linux-headers/uapi/.gitignore

diff --git a/devtools/check-linux-uapi.sh b/devtools/check-linux-uapi.sh
new file mode 100755
index 0000000000..76111d78ce
--- /dev/null
+++ b/devtools/check-linux-uapi.sh
@@ -0,0 +1,74 @@
+#!/bin/sh
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2024 Red Hat, Inc.
+
+#
+# Import Linux Kernel uAPI header file
+#
+
+base_url="https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/include/uapi/"
+base_path="linux-headers/uapi/"
+errors=0
+
+print_usage()
+{
+	echo "Usage: $(basename $0) [-h]"
+}
+
+check_uapi_header() {
+	path=$1
+	file=${1//"$base_path"/}
+	version=$(git log --format=%b -1 $path | sed -ne 's/^uAPI Version: \(.*\)$/\1/p')
+
+	url="${base_url}${file}?h=${version}"
+	echo -n "Checking $file for version $version... "
+	curl -s -f -o $tmpinput $url
+	if [ $? -ne 0 ]; then
+		echo "Failed to download $url"
+		exit 1
+	fi
+
+	diff -q $path $tmpinput >/dev/null
+	if [ $? -ne 0 ]; then
+		echo "KO"
+		diff -u $path $tmpinput
+		errors=$((errors+1))
+	else
+		echo "OK"
+	fi
+}
+
+while getopts hv ARG ; do
+	case $ARG in
+		h )
+			print_usage
+			exit 0
+			;;
+		? )
+			print_usage
+			exit 1
+			;;
+	esac
+done
+
+shift $(($OPTIND - 1))
+if [ $# -ne 0 ]; then
+	print_usage
+	exit 1
+fi
+
+cd $(dirname $0)/..
+
+tmpinput=$(mktemp -t dpdk.checkuapi.XXXXXX)
+trap "rm -f '$tmpinput'" INT
+
+while IFS= read -d '' -r filename; do
+	check_uapi_header "${filename}" </dev/null
+done < <(find $base_path -name "*.h" -type f -print0)
+
+echo "$errors error(s) found"
+
+rm -f $tmpinput
+trap - INT
+
+exit $errors
diff --git a/devtools/import-linux-uapi.sh b/devtools/import-linux-uapi.sh
new file mode 100755
index 0000000000..e48dc6546c
--- /dev/null
+++ b/devtools/import-linux-uapi.sh
@@ -0,0 +1,45 @@
+#!/bin/sh -e
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2024 Red Hat, Inc.
+
+#
+# Import Linux Kernel uAPI header file
+#
+
+base_url="https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/include/uapi/"
+base_path="linux-headers/uapi/"
+
+print_usage()
+{
+	echo "Usage: $(basename $0) [-h] file version"
+	echo "Example of valid file: linux/vfio.h"
+	echo "Example of valid version: v6.10"
+}
+
+while getopts hv ARG ; do
+	case $ARG in
+		h )
+			print_usage
+			exit 0
+			;;
+		? )
+			print_usage
+			exit 1
+			;;
+	esac
+done
+shift $(($OPTIND - 1))
+
+if [ $# -ne 2 ]; then
+	print_usage
+	exit 1
+fi
+
+file=$1
+version=$2
+
+url="${base_url}${file}?h=${version}"
+path="${base_path}${file}"
+
+cd $(dirname $0)/..
+curl -s -f --create-dirs -o $path $url
diff --git a/doc/guides/contributing/index.rst b/doc/guides/contributing/index.rst
index dcb9b1fbf0..603dc72654 100644
--- a/doc/guides/contributing/index.rst
+++ b/doc/guides/contributing/index.rst
@@ -19,3 +19,4 @@ Contributor's Guidelines
     vulnerability
     stable
     cheatsheet
+    linux_uapi
diff --git a/doc/guides/contributing/linux_uapi.rst b/doc/guides/contributing/linux_uapi.rst
new file mode 100644
index 0000000000..a3f684013a
--- /dev/null
+++ b/doc/guides/contributing/linux_uapi.rst
@@ -0,0 +1,77 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+   Copyright(c) 2024 Red Hat, Inc.
+
+Linux uAPI header files
+=======================
+
+Rationale
+---------
+
+The system a DPDK library or driver is built on is not necessarily running the
+same Kernel version than the system that will run it.
+Importing Linux Kernel uAPI headers enable to build features that are not
+supported yet by the build system.
+
+For example, the build system runs upstream Kernel v5.19 and we would like to
+build a VDUSE application that will use VDUSE_IOTLB_GET_INFO ioctl() introduced
+in Linux Kernel v6.0.
+
+`Linux Kernel licence exception regarding syscalls
+<https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/LICENSES/exceptions/Linux-syscall-note>`_
+enable importing unmodified Linux Kernel uAPI header files.
+
+Importing or updating an uAPI header file
+-----------------------------------------
+
+In order to ensure the imported uAPI headers are both unmodified and from a
+released version of the linux Kernel, a helper script is made available and
+MUST be used.
+Below is an example to import ``linux/vduse.h`` file from Linux ``v6.10``:
+
+.. code-block:: console
+
+   ./devtools/import-linux-uapi.sh linux/vduse.h v6.10
+
+Once imported, the header files should be committed without any other change,
+and the commit message MUST specify the imported version using
+``uAPI Version:`` tag and title MUST be prefixed with uapi keyword.
+For example::
+
+  uapi: import VDUSE header file
+
+  This patch imports VDUSE uAPI header file for inclusion
+  into the Vhost library.
+
+  uAPI Version: v6.10
+
+  Signed-off-by: Alex Smith <alex.smith@example.com>
+
+Updating an already imported header to a newer released version should only
+be done on a need basis.
+The commit message should reflect why updating the header is necessary.
+
+Once committed, user can check headers and commit message are valid by using
+the Linux uAPI checker tool:
+
+.. code-block:: console
+
+   ./devtools/check-linux-uapi.sh
+
+Header inclusion into library or driver
+---------------------------------------
+
+The library or driver willing to make use of imported uAPI headers needs to
+explicitly add uAPI headers path to the ``includes`` var in its ``meson.build``
+file:
+
+.. code-block:: python
+
+   includes += linux_uapi_inc
+
+Then, it can be included with ``uapi/`` prefix in C files.
+For example to include VDUSE uAPI:
+
+.. code-block:: c
+
+   #include <uapi/linux/vduse.h>
+
diff --git a/linux-headers/uapi/.gitignore b/linux-headers/uapi/.gitignore
new file mode 100644
index 0000000000..88829d04e9
--- /dev/null
+++ b/linux-headers/uapi/.gitignore
@@ -0,0 +1,3 @@
+**
+!**/
+!**/*.h
diff --git a/meson.build b/meson.build
index 8b248d4505..aca47334ab 100644
--- a/meson.build
+++ b/meson.build
@@ -67,16 +67,20 @@ endif
 # configure the build, and make sure configs here and in config folder are
 # able to be included in any file. We also store a global array of include dirs
 # for passing to pmdinfogen scripts
-global_inc = include_directories('.', 'config',
+global_inc = [include_directories('.', 'config',
     'lib/eal/include',
     'lib/eal/@0@/include'.format(host_machine.system()),
     'lib/eal/@0@/include'.format(arch_subdir),
-)
+)]
 
 # do configuration and get tool paths
 subdir('buildtools')
 subdir('config')
 
+if is_linux
+    global_inc += include_directories('linux-headers')
+endif
+
 # build libs and drivers
 subdir('lib')
 subdir('drivers')
-- 
2.46.0

[RFC v2 2/3] uapi: import VDUSE header

[Maxime Coquelin]

uAPI Version: v6.10

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
 linux-headers/uapi/linux/vduse.h | 353 +++++++++++++++++++++++++++++++
 1 file changed, 353 insertions(+)
 create mode 100644 linux-headers/uapi/linux/vduse.h

diff --git a/linux-headers/uapi/linux/vduse.h b/linux-headers/uapi/linux/vduse.h
new file mode 100644
index 0000000000..11bd48c72c
--- /dev/null
+++ b/linux-headers/uapi/linux/vduse.h
@@ -0,0 +1,353 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _UAPI_VDUSE_H_
+#define _UAPI_VDUSE_H_
+
+#include <linux/types.h>
+
+#define VDUSE_BASE	0x81
+
+/* The ioctls for control device (/dev/vduse/control) */
+
+#define VDUSE_API_VERSION	0
+
+/*
+ * Get the version of VDUSE API that kernel supported (VDUSE_API_VERSION).
+ * This is used for future extension.
+ */
+#define VDUSE_GET_API_VERSION	_IOR(VDUSE_BASE, 0x00, __u64)
+
+/* Set the version of VDUSE API that userspace supported. */
+#define VDUSE_SET_API_VERSION	_IOW(VDUSE_BASE, 0x01, __u64)
+
+/**
+ * struct vduse_dev_config - basic configuration of a VDUSE device
+ * @name: VDUSE device name, needs to be NUL terminated
+ * @vendor_id: virtio vendor id
+ * @device_id: virtio device id
+ * @features: virtio features
+ * @vq_num: the number of virtqueues
+ * @vq_align: the allocation alignment of virtqueue's metadata
+ * @reserved: for future use, needs to be initialized to zero
+ * @config_size: the size of the configuration space
+ * @config: the buffer of the configuration space
+ *
+ * Structure used by VDUSE_CREATE_DEV ioctl to create VDUSE device.
+ */
+struct vduse_dev_config {
+#define VDUSE_NAME_MAX	256
+	char name[VDUSE_NAME_MAX];
+	__u32 vendor_id;
+	__u32 device_id;
+	__u64 features;
+	__u32 vq_num;
+	__u32 vq_align;
+	__u32 reserved[13];
+	__u32 config_size;
+	__u8 config[];
+};
+
+/* Create a VDUSE device which is represented by a char device (/dev/vduse/$NAME) */
+#define VDUSE_CREATE_DEV	_IOW(VDUSE_BASE, 0x02, struct vduse_dev_config)
+
+/*
+ * Destroy a VDUSE device. Make sure there are no more references
+ * to the char device (/dev/vduse/$NAME).
+ */
+#define VDUSE_DESTROY_DEV	_IOW(VDUSE_BASE, 0x03, char[VDUSE_NAME_MAX])
+
+/* The ioctls for VDUSE device (/dev/vduse/$NAME) */
+
+/**
+ * struct vduse_iotlb_entry - entry of IOTLB to describe one IOVA region [start, last]
+ * @offset: the mmap offset on returned file descriptor
+ * @start: start of the IOVA region
+ * @last: last of the IOVA region
+ * @perm: access permission of the IOVA region
+ *
+ * Structure used by VDUSE_IOTLB_GET_FD ioctl to find an overlapped IOVA region.
+ */
+struct vduse_iotlb_entry {
+	__u64 offset;
+	__u64 start;
+	__u64 last;
+#define VDUSE_ACCESS_RO 0x1
+#define VDUSE_ACCESS_WO 0x2
+#define VDUSE_ACCESS_RW 0x3
+	__u8 perm;
+};
+
+/*
+ * Find the first IOVA region that overlaps with the range [start, last]
+ * and return the corresponding file descriptor. Return -EINVAL means the
+ * IOVA region doesn't exist. Caller should set start and last fields.
+ */
+#define VDUSE_IOTLB_GET_FD	_IOWR(VDUSE_BASE, 0x10, struct vduse_iotlb_entry)
+
+/*
+ * Get the negotiated virtio features. It's a subset of the features in
+ * struct vduse_dev_config which can be accepted by virtio driver. It's
+ * only valid after FEATURES_OK status bit is set.
+ */
+#define VDUSE_DEV_GET_FEATURES	_IOR(VDUSE_BASE, 0x11, __u64)
+
+/**
+ * struct vduse_config_data - data used to update configuration space
+ * @offset: the offset from the beginning of configuration space
+ * @length: the length to write to configuration space
+ * @buffer: the buffer used to write from
+ *
+ * Structure used by VDUSE_DEV_SET_CONFIG ioctl to update device
+ * configuration space.
+ */
+struct vduse_config_data {
+	__u32 offset;
+	__u32 length;
+	__u8 buffer[];
+};
+
+/* Set device configuration space */
+#define VDUSE_DEV_SET_CONFIG	_IOW(VDUSE_BASE, 0x12, struct vduse_config_data)
+
+/*
+ * Inject a config interrupt. It's usually used to notify virtio driver
+ * that device configuration space has changed.
+ */
+#define VDUSE_DEV_INJECT_CONFIG_IRQ	_IO(VDUSE_BASE, 0x13)
+
+/**
+ * struct vduse_vq_config - basic configuration of a virtqueue
+ * @index: virtqueue index
+ * @max_size: the max size of virtqueue
+ * @reserved: for future use, needs to be initialized to zero
+ *
+ * Structure used by VDUSE_VQ_SETUP ioctl to setup a virtqueue.
+ */
+struct vduse_vq_config {
+	__u32 index;
+	__u16 max_size;
+	__u16 reserved[13];
+};
+
+/*
+ * Setup the specified virtqueue. Make sure all virtqueues have been
+ * configured before the device is attached to vDPA bus.
+ */
+#define VDUSE_VQ_SETUP		_IOW(VDUSE_BASE, 0x14, struct vduse_vq_config)
+
+/**
+ * struct vduse_vq_state_split - split virtqueue state
+ * @avail_index: available index
+ */
+struct vduse_vq_state_split {
+	__u16 avail_index;
+};
+
+/**
+ * struct vduse_vq_state_packed - packed virtqueue state
+ * @last_avail_counter: last driver ring wrap counter observed by device
+ * @last_avail_idx: device available index
+ * @last_used_counter: device ring wrap counter
+ * @last_used_idx: used index
+ */
+struct vduse_vq_state_packed {
+	__u16 last_avail_counter;
+	__u16 last_avail_idx;
+	__u16 last_used_counter;
+	__u16 last_used_idx;
+};
+
+/**
+ * struct vduse_vq_info - information of a virtqueue
+ * @index: virtqueue index
+ * @num: the size of virtqueue
+ * @desc_addr: address of desc area
+ * @driver_addr: address of driver area
+ * @device_addr: address of device area
+ * @split: split virtqueue state
+ * @packed: packed virtqueue state
+ * @ready: ready status of virtqueue
+ *
+ * Structure used by VDUSE_VQ_GET_INFO ioctl to get virtqueue's information.
+ */
+struct vduse_vq_info {
+	__u32 index;
+	__u32 num;
+	__u64 desc_addr;
+	__u64 driver_addr;
+	__u64 device_addr;
+	union {
+		struct vduse_vq_state_split split;
+		struct vduse_vq_state_packed packed;
+	};
+	__u8 ready;
+};
+
+/* Get the specified virtqueue's information. Caller should set index field. */
+#define VDUSE_VQ_GET_INFO	_IOWR(VDUSE_BASE, 0x15, struct vduse_vq_info)
+
+/**
+ * struct vduse_vq_eventfd - eventfd configuration for a virtqueue
+ * @index: virtqueue index
+ * @fd: eventfd, -1 means de-assigning the eventfd
+ *
+ * Structure used by VDUSE_VQ_SETUP_KICKFD ioctl to setup kick eventfd.
+ */
+struct vduse_vq_eventfd {
+	__u32 index;
+#define VDUSE_EVENTFD_DEASSIGN -1
+	int fd;
+};
+
+/*
+ * Setup kick eventfd for specified virtqueue. The kick eventfd is used
+ * by VDUSE kernel module to notify userspace to consume the avail vring.
+ */
+#define VDUSE_VQ_SETUP_KICKFD	_IOW(VDUSE_BASE, 0x16, struct vduse_vq_eventfd)
+
+/*
+ * Inject an interrupt for specific virtqueue. It's used to notify virtio driver
+ * to consume the used vring.
+ */
+#define VDUSE_VQ_INJECT_IRQ	_IOW(VDUSE_BASE, 0x17, __u32)
+
+/**
+ * struct vduse_iova_umem - userspace memory configuration for one IOVA region
+ * @uaddr: start address of userspace memory, it must be aligned to page size
+ * @iova: start of the IOVA region
+ * @size: size of the IOVA region
+ * @reserved: for future use, needs to be initialized to zero
+ *
+ * Structure used by VDUSE_IOTLB_REG_UMEM and VDUSE_IOTLB_DEREG_UMEM
+ * ioctls to register/de-register userspace memory for IOVA regions
+ */
+struct vduse_iova_umem {
+	__u64 uaddr;
+	__u64 iova;
+	__u64 size;
+	__u64 reserved[3];
+};
+
+/* Register userspace memory for IOVA regions */
+#define VDUSE_IOTLB_REG_UMEM	_IOW(VDUSE_BASE, 0x18, struct vduse_iova_umem)
+
+/* De-register the userspace memory. Caller should set iova and size field. */
+#define VDUSE_IOTLB_DEREG_UMEM	_IOW(VDUSE_BASE, 0x19, struct vduse_iova_umem)
+
+/**
+ * struct vduse_iova_info - information of one IOVA region
+ * @start: start of the IOVA region
+ * @last: last of the IOVA region
+ * @capability: capability of the IOVA regsion
+ * @reserved: for future use, needs to be initialized to zero
+ *
+ * Structure used by VDUSE_IOTLB_GET_INFO ioctl to get information of
+ * one IOVA region.
+ */
+struct vduse_iova_info {
+	__u64 start;
+	__u64 last;
+#define VDUSE_IOVA_CAP_UMEM (1 << 0)
+	__u64 capability;
+	__u64 reserved[3];
+};
+
+/*
+ * Find the first IOVA region that overlaps with the range [start, last]
+ * and return some information on it. Caller should set start and last fields.
+ */
+#define VDUSE_IOTLB_GET_INFO	_IOWR(VDUSE_BASE, 0x1a, struct vduse_iova_info)
+
+/* The control messages definition for read(2)/write(2) on /dev/vduse/$NAME */
+
+/**
+ * enum vduse_req_type - request type
+ * @VDUSE_GET_VQ_STATE: get the state for specified virtqueue from userspace
+ * @VDUSE_SET_STATUS: set the device status
+ * @VDUSE_UPDATE_IOTLB: Notify userspace to update the memory mapping for
+ *                      specified IOVA range via VDUSE_IOTLB_GET_FD ioctl
+ */
+enum vduse_req_type {
+	VDUSE_GET_VQ_STATE,
+	VDUSE_SET_STATUS,
+	VDUSE_UPDATE_IOTLB,
+};
+
+/**
+ * struct vduse_vq_state - virtqueue state
+ * @index: virtqueue index
+ * @split: split virtqueue state
+ * @packed: packed virtqueue state
+ */
+struct vduse_vq_state {
+	__u32 index;
+	union {
+		struct vduse_vq_state_split split;
+		struct vduse_vq_state_packed packed;
+	};
+};
+
+/**
+ * struct vduse_dev_status - device status
+ * @status: device status
+ */
+struct vduse_dev_status {
+	__u8 status;
+};
+
+/**
+ * struct vduse_iova_range - IOVA range [start, last]
+ * @start: start of the IOVA range
+ * @last: last of the IOVA range
+ */
+struct vduse_iova_range {
+	__u64 start;
+	__u64 last;
+};
+
+/**
+ * struct vduse_dev_request - control request
+ * @type: request type
+ * @request_id: request id
+ * @reserved: for future use
+ * @vq_state: virtqueue state, only index field is available
+ * @s: device status
+ * @iova: IOVA range for updating
+ * @padding: padding
+ *
+ * Structure used by read(2) on /dev/vduse/$NAME.
+ */
+struct vduse_dev_request {
+	__u32 type;
+	__u32 request_id;
+	__u32 reserved[4];
+	union {
+		struct vduse_vq_state vq_state;
+		struct vduse_dev_status s;
+		struct vduse_iova_range iova;
+		__u32 padding[32];
+	};
+};
+
+/**
+ * struct vduse_dev_response - response to control request
+ * @request_id: corresponding request id
+ * @result: the result of request
+ * @reserved: for future use, needs to be initialized to zero
+ * @vq_state: virtqueue state
+ * @padding: padding
+ *
+ * Structure used by write(2) on /dev/vduse/$NAME.
+ */
+struct vduse_dev_response {
+	__u32 request_id;
+#define VDUSE_REQ_RESULT_OK	0x00
+#define VDUSE_REQ_RESULT_FAILED	0x01
+	__u32 result;
+	__u32 reserved[4];
+	union {
+		struct vduse_vq_state vq_state;
+		__u32 padding[32];
+	};
+};
+
+#endif /* _UAPI_VDUSE_H_ */
-- 
2.46.0

[RFC v2 3/3] vduse: use import VDUSE uAPI header

[Maxime Coquelin]

This patch makes use of the imported VDUSE headers.
The VDUSE support is now systematically built on Linux
systems, even if the build system does not support its
ioctl().

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---
 lib/vhost/meson.build |  5 +----
 lib/vhost/vduse.c     |  2 +-
 lib/vhost/vduse.h     | 22 ----------------------
 3 files changed, 2 insertions(+), 27 deletions(-)

diff --git a/lib/vhost/meson.build b/lib/vhost/meson.build
index 41b622a9be..5c2528ff8c 100644
--- a/lib/vhost/meson.build
+++ b/lib/vhost/meson.build
@@ -23,16 +23,13 @@ sources = files(
         'iotlb.c',
         'socket.c',
         'vdpa.c',
+        'vduse.c',
         'vhost.c',
         'vhost_crypto.c',
         'vhost_user.c',
         'virtio_net.c',
         'virtio_net_ctrl.c',
 )
-if cc.has_header('linux/vduse.h')
-    sources += files('vduse.c')
-    cflags += '-DVHOST_HAS_VDUSE'
-endif
 headers = files(
         'rte_vdpa.h',
         'rte_vhost.h',
diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c
index c66602905c..a41ff2024c 100644
--- a/lib/vhost/vduse.c
+++ b/lib/vhost/vduse.c
@@ -8,7 +8,7 @@
 #include <fcntl.h>
 
 
-#include <linux/vduse.h>
+#include <uapi/linux/vduse.h>
 #include <linux/virtio_net.h>
 
 #include <sys/ioctl.h>
diff --git a/lib/vhost/vduse.h b/lib/vhost/vduse.h
index 0d8f3f1205..47ca97a064 100644
--- a/lib/vhost/vduse.h
+++ b/lib/vhost/vduse.h
@@ -9,29 +9,7 @@
 
 #define VDUSE_NET_SUPPORTED_FEATURES VIRTIO_NET_SUPPORTED_FEATURES
 
-#ifdef VHOST_HAS_VDUSE
-
 int vduse_device_create(const char *path, bool compliant_ol_flags);
 int vduse_device_destroy(const char *path);
 
-#else
-
-static inline int
-vduse_device_create(const char *path, bool compliant_ol_flags)
-{
-	RTE_SET_USED(compliant_ol_flags);
-
-	VHOST_CONFIG_LOG(path, ERR, "VDUSE support disabled at build time");
-	return -1;
-}
-
-static inline int
-vduse_device_destroy(const char *path)
-{
-	VHOST_CONFIG_LOG(path, ERR, "VDUSE support disabled at build time");
-	return -1;
-}
-
-#endif /* VHOST_HAS_VDUSE */
-
 #endif /* _VDUSE_H */
-- 
2.46.0

RE: [RFC v2 1/3] uapi: introduce kernel uAPI headers import

[Morten Brørup]

> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com]
> Sent: Friday, 6 September 2024 17.24
> 
> This patch introduces uAPI headers import into the DPDK
> repository. This import is possible thanks to Linux Kernel
> licence exception for syscalls:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/
> LICENSES/exceptions/Linux-syscall-note
> 
> Header files are have to be explicitly imported, and
> libraries and drivers have to explicitly enable their
> inclusion.
> 
> Guidelines are provided in the documentation, and helper
> scripts are also provided to ensure proper importation of the
> header (unmodified content from a released Kernel version):
>  - import-linux-uapi.sh: used to download and install a given
> header to linux-headers/uapi/
>  - check-linux-uapi.sh: used to check all headers are valid
> 
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---

Acked-by: Morten Brørup <mb@smartsharesystems.com>

Re: [RFC v2 1/3] uapi: introduce kernel uAPI headers import

[David Marchand]

On Fri, Sep 6, 2024 at 5:23 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:

[snip]

> diff --git a/devtools/check-linux-uapi.sh b/devtools/check-linux-uapi.sh
> new file mode 100755
> index 0000000000..76111d78ce
> --- /dev/null
> +++ b/devtools/check-linux-uapi.sh
> @@ -0,0 +1,74 @@
> +#!/bin/sh

-e maybe?

> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright (c) 2024 Red Hat, Inc.
> +
> +#
> +# Import Linux Kernel uAPI header file

# Check Linux Kernel uAPI headers

> +#
> +
> +base_url="https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/include/uapi/"
> +base_path="linux-headers/uapi/"
> +errors=0
> +
> +print_usage()
> +{
> +       echo "Usage: $(basename $0) [-h]"
> +}
> +
> +check_uapi_header() {
> +       path=$1
> +       file=${1//"$base_path"/}

I suspect it is a bashism.

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html
${parameter#[word]}
Remove Smallest Prefix Pattern. The word shall be expanded to produce
a pattern. The parameter expansion shall then result in parameter,
with the smallest portion of the prefix matched by the pattern
deleted. If present, word shall not begin with an unquoted '#'.

So a POSIX alternative is:
file=${1#$base_path/}


> +       version=$(git log --format=%b -1 $path | sed -ne 's/^uAPI Version: \(.*\)$/\1/p')

I would add a check and raise a warning (error?) if extracting version
fails (iow -z "$version").


> +
> +       url="${base_url}${file}?h=${version}"
> +       echo -n "Checking $file for version $version... "
> +       curl -s -f -o $tmpinput $url
> +       if [ $? -ne 0 ]; then
> +               echo "Failed to download $url"
> +               exit 1
> +       fi
> +
> +       diff -q $path $tmpinput >/dev/null
> +       if [ $? -ne 0 ]; then
> +               echo "KO"
> +               diff -u $path $tmpinput
> +               errors=$((errors+1))
> +       else
> +               echo "OK"
> +       fi
> +}
> +
> +while getopts hv ARG ; do
> +       case $ARG in
> +               h )
> +                       print_usage
> +                       exit 0
> +                       ;;
> +               ? )
> +                       print_usage
> +                       exit 1
> +                       ;;
> +       esac
> +done
> +
> +shift $(($OPTIND - 1))
> +if [ $# -ne 0 ]; then
> +       print_usage
> +       exit 1
> +fi
> +
> +cd $(dirname $0)/..
> +
> +tmpinput=$(mktemp -t dpdk.checkuapi.XXXXXX)
> +trap "rm -f '$tmpinput'" INT
> +
> +while IFS= read -d '' -r filename; do

Simpler:

for filename in $(find $base_path -name "*.h" -type f); do
     check_uapi_header "${filename}" </dev/null
done

> +       check_uapi_header "${filename}" </dev/null
> +done < <(find $base_path -name "*.h" -type f -print0)


> +
> +echo "$errors error(s) found"
> +
> +rm -f $tmpinput
> +trap - INT
> +
> +exit $errors

[snip]

> diff --git a/doc/guides/contributing/linux_uapi.rst b/doc/guides/contributing/linux_uapi.rst
> new file mode 100644
> index 0000000000..a3f684013a
> --- /dev/null
> +++ b/doc/guides/contributing/linux_uapi.rst
> @@ -0,0 +1,77 @@
> +.. SPDX-License-Identifier: BSD-3-Clause
> +   Copyright(c) 2024 Red Hat, Inc.
> +
> +Linux uAPI header files
> +=======================
> +
> +Rationale
> +---------
> +
> +The system a DPDK library or driver is built on is not necessarily running the
> +same Kernel version than the system that will run it.
> +Importing Linux Kernel uAPI headers enable to build features that are not
> +supported yet by the build system.
> +
> +For example, the build system runs upstream Kernel v5.19 and we would like to
> +build a VDUSE application that will use VDUSE_IOTLB_GET_INFO ioctl() introduced
> +in Linux Kernel v6.0.
> +
> +`Linux Kernel licence exception regarding syscalls
> +<https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/LICENSES/exceptions/Linux-syscall-note>`_
> +enable importing unmodified Linux Kernel uAPI header files.
> +
> +Importing or updating an uAPI header file
> +-----------------------------------------
> +
> +In order to ensure the imported uAPI headers are both unmodified and from a
> +released version of the linux Kernel, a helper script is made available and
> +MUST be used.
> +Below is an example to import ``linux/vduse.h`` file from Linux ``v6.10``:
> +
> +.. code-block:: console
> +
> +   ./devtools/import-linux-uapi.sh linux/vduse.h v6.10
> +
> +Once imported, the header files should be committed without any other change,
> +and the commit message MUST specify the imported version using
> +``uAPI Version:`` tag and title MUST be prefixed with uapi keyword.
> +For example::
> +
> +  uapi: import VDUSE header file
> +
> +  This patch imports VDUSE uAPI header file for inclusion
> +  into the Vhost library.
> +
> +  uAPI Version: v6.10
> +
> +  Signed-off-by: Alex Smith <alex.smith@example.com>
> +
> +Updating an already imported header to a newer released version should only
> +be done on a need basis.
> +The commit message should reflect why updating the header is necessary.

+1.


> +
> +Once committed, user can check headers and commit message are valid by using
> +the Linux uAPI checker tool:
> +
> +.. code-block:: console
> +
> +   ./devtools/check-linux-uapi.sh

And thanks for adding this check.
It will help maintainers.


> +
> +Header inclusion into library or driver
> +---------------------------------------
> +
> +The library or driver willing to make use of imported uAPI headers needs to
> +explicitly add uAPI headers path to the ``includes`` var in its ``meson.build``
> +file:
> +
> +.. code-block:: python
> +
> +   includes += linux_uapi_inc

Now that the uapi headers directory is pushed to global_inc, there is
no need for this part in the doc.


> +
> +Then, it can be included with ``uapi/`` prefix in C files.
> +For example to include VDUSE uAPI:
> +
> +.. code-block:: c
> +
> +   #include <uapi/linux/vduse.h>
> +
> diff --git a/linux-headers/uapi/.gitignore b/linux-headers/uapi/.gitignore
> new file mode 100644
> index 0000000000..88829d04e9
> --- /dev/null
> +++ b/linux-headers/uapi/.gitignore
> @@ -0,0 +1,3 @@
> +**
> +!**/
> +!**/*.h

Nice trick to solve the per patch build issue :-).


-- 
David Marchand

Re: [RFC v2 0/3] Import Kernel uAPI header files

[Stephen Hemminger]

On Fri,  6 Sep 2024 17:23:34 +0200
Maxime Coquelin <maxime.coquelin@redhat.com> wrote:

> This series enables importing Linux Kernel uAPI headers
> into the DPDK repository. It aims at solving alignment
> issues between the build system and the system applications
> linked ot DPDK libraries are run on.
> 
> It can also help simplify spaghetti code done to support
> different versions of the Linux Kernel headers used by
> the build system.
> 
> Guidelines and import script are also part of first patch.
> A uAPI checker script is added in this 2nd RFC, goals is to
> use it in CI for any patch touching linux-headers/uapi.
> 
> Follow-up patches are an example of imported uAPI inclusion
> of VDUSE header into the Vhost library.
> 
> Morten, I did not apply your Ack on patch 1, as it has
> some significant changes and additions.

I know of build systems are done in containers and are intentionally setup
to not allow access to the public Internet. You will break them.

Re: [RFC v2 0/3] Import Kernel uAPI header files

[Maxime Coquelin]

On 9/9/24 02:31, Stephen Hemminger wrote:
> On Fri,  6 Sep 2024 17:23:34 +0200
> Maxime Coquelin <maxime.coquelin@redhat.com> wrote:
> 
>> This series enables importing Linux Kernel uAPI headers
>> into the DPDK repository. It aims at solving alignment
>> issues between the build system and the system applications
>> linked ot DPDK libraries are run on.
>>
>> It can also help simplify spaghetti code done to support
>> different versions of the Linux Kernel headers used by
>> the build system.
>>
>> Guidelines and import script are also part of first patch.
>> A uAPI checker script is added in this 2nd RFC, goals is to
>> use it in CI for any patch touching linux-headers/uapi.
>>
>> Follow-up patches are an example of imported uAPI inclusion
>> of VDUSE header into the Vhost library.
>>
>> Morten, I did not apply your Ack on patch 1, as it has
>> some significant changes and additions.
> 
> I know of build systems are done in containers and are intentionally setup
> to not allow access to the public Internet. You will break them.
> 

It is not expected to be run by the build systems.
Only new header import and check scripts are requiring access to the
public internet.

Re: [RFC v2 1/3] uapi: introduce kernel uAPI headers import

[Maxime Coquelin]

On 9/7/24 16:34, David Marchand wrote:
> On Fri, Sep 6, 2024 at 5:23 PM Maxime Coquelin
> <maxime.coquelin@redhat.com> wrote:
> 
> [snip]
> 
>> diff --git a/devtools/check-linux-uapi.sh b/devtools/check-linux-uapi.sh
>> new file mode 100755
>> index 0000000000..76111d78ce
>> --- /dev/null
>> +++ b/devtools/check-linux-uapi.sh
>> @@ -0,0 +1,74 @@
>> +#!/bin/sh
> 
> -e maybe?

It will stop on first diff without reporting anything.
At least on how it is done for now.

>> +# SPDX-License-Identifier: BSD-3-Clause
>> +# Copyright (c) 2024 Red Hat, Inc.
>> +
>> +#
>> +# Import Linux Kernel uAPI header file
> 
> # Check Linux Kernel uAPI headers
> 
>> +#
>> +
>> +base_url="https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/include/uapi/"
>> +base_path="linux-headers/uapi/"
>> +errors=0
>> +
>> +print_usage()
>> +{
>> +       echo "Usage: $(basename $0) [-h]"
>> +}
>> +
>> +check_uapi_header() {
>> +       path=$1
>> +       file=${1//"$base_path"/}
> 
> I suspect it is a bashism.
> 
> https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html
> ${parameter#[word]}
> Remove Smallest Prefix Pattern. The word shall be expanded to produce
> a pattern. The parameter expansion shall then result in parameter,
> with the smallest portion of the prefix matched by the pattern
> deleted. If present, word shall not begin with an unquoted '#'.
> 
> So a POSIX alternative is:
> file=${1#$base_path/}

Changed to:
file=${1#$base_path}

> 
> 
>> +       version=$(git log --format=%b -1 $path | sed -ne 's/^uAPI Version: \(.*\)$/\1/p')
> 
> I would add a check and raise a warning (error?) if extracting version
> fails (iow -z "$version").

It disappears in new version.

> 
>> +
>> +       url="${base_url}${file}?h=${version}"
>> +       echo -n "Checking $file for version $version... "
>> +       curl -s -f -o $tmpinput $url
>> +       if [ $? -ne 0 ]; then
>> +               echo "Failed to download $url"
>> +               exit 1
>> +       fi
>> +
>> +       diff -q $path $tmpinput >/dev/null
>> +       if [ $? -ne 0 ]; then
>> +               echo "KO"
>> +               diff -u $path $tmpinput
>> +               errors=$((errors+1))
>> +       else
>> +               echo "OK"
>> +       fi
>> +}
>> +
>> +while getopts hv ARG ; do
>> +       case $ARG in
>> +               h )
>> +                       print_usage
>> +                       exit 0
>> +                       ;;
>> +               ? )
>> +                       print_usage
>> +                       exit 1
>> +                       ;;
>> +       esac
>> +done
>> +
>> +shift $(($OPTIND - 1))
>> +if [ $# -ne 0 ]; then
>> +       print_usage
>> +       exit 1
>> +fi
>> +
>> +cd $(dirname $0)/..
>> +
>> +tmpinput=$(mktemp -t dpdk.checkuapi.XXXXXX)
>> +trap "rm -f '$tmpinput'" INT
>> +
>> +while IFS= read -d '' -r filename; do
> 
> Simpler:
> 
> for filename in $(find $base_path -name "*.h" -type f); do
>       check_uapi_header "${filename}" </dev/null
> done

Done!

> 
>> +       check_uapi_header "${filename}" </dev/null
>> +done < <(find $base_path -name "*.h" -type f -print0)
> 
> 
>> +
>> +echo "$errors error(s) found"
>> +
>> +rm -f $tmpinput
>> +trap - INT
>> +
>> +exit $errors
> 
> [snip]
> 
>> diff --git a/doc/guides/contributing/linux_uapi.rst b/doc/guides/contributing/linux_uapi.rst
>> new file mode 100644
>> index 0000000000..a3f684013a
>> --- /dev/null
>> +++ b/doc/guides/contributing/linux_uapi.rst
>> @@ -0,0 +1,77 @@
>> +.. SPDX-License-Identifier: BSD-3-Clause
>> +   Copyright(c) 2024 Red Hat, Inc.
>> +
>> +Linux uAPI header files
>> +=======================
>> +
>> +Rationale
>> +---------
>> +
>> +The system a DPDK library or driver is built on is not necessarily running the
>> +same Kernel version than the system that will run it.
>> +Importing Linux Kernel uAPI headers enable to build features that are not
>> +supported yet by the build system.
>> +
>> +For example, the build system runs upstream Kernel v5.19 and we would like to
>> +build a VDUSE application that will use VDUSE_IOTLB_GET_INFO ioctl() introduced
>> +in Linux Kernel v6.0.
>> +
>> +`Linux Kernel licence exception regarding syscalls
>> +<https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/LICENSES/exceptions/Linux-syscall-note>`_
>> +enable importing unmodified Linux Kernel uAPI header files.
>> +
>> +Importing or updating an uAPI header file
>> +-----------------------------------------
>> +
>> +In order to ensure the imported uAPI headers are both unmodified and from a
>> +released version of the linux Kernel, a helper script is made available and
>> +MUST be used.
>> +Below is an example to import ``linux/vduse.h`` file from Linux ``v6.10``:
>> +
>> +.. code-block:: console
>> +
>> +   ./devtools/import-linux-uapi.sh linux/vduse.h v6.10
>> +
>> +Once imported, the header files should be committed without any other change,
>> +and the commit message MUST specify the imported version using
>> +``uAPI Version:`` tag and title MUST be prefixed with uapi keyword.
>> +For example::
>> +
>> +  uapi: import VDUSE header file
>> +
>> +  This patch imports VDUSE uAPI header file for inclusion
>> +  into the Vhost library.
>> +
>> +  uAPI Version: v6.10
>> +
>> +  Signed-off-by: Alex Smith <alex.smith@example.com>
>> +
>> +Updating an already imported header to a newer released version should only
>> +be done on a need basis.
>> +The commit message should reflect why updating the header is necessary.
> 
> +1.
> 
> 
>> +
>> +Once committed, user can check headers and commit message are valid by using
>> +the Linux uAPI checker tool:
>> +
>> +.. code-block:: console
>> +
>> +   ./devtools/check-linux-uapi.sh
> 
> And thanks for adding this check.
> It will help maintainers.

Welcome.
Do you think we would need a sub-maintainer for this, or you prefere the
main maintainers to take care of this?

I'll be happy to help with the maintenance if you feel the need.

> 
>> +
>> +Header inclusion into library or driver
>> +---------------------------------------
>> +
>> +The library or driver willing to make use of imported uAPI headers needs to
>> +explicitly add uAPI headers path to the ``includes`` var in its ``meson.build``
>> +file:
>> +
>> +.. code-block:: python
>> +
>> +   includes += linux_uapi_inc
> 
> Now that the uapi headers directory is pushed to global_inc, there is
> no need for this part in the doc.

Right, I'm also adapting the commit message to reflect this.

> 
>> +
>> +Then, it can be included with ``uapi/`` prefix in C files.
>> +For example to include VDUSE uAPI:
>> +
>> +.. code-block:: c
>> +
>> +   #include <uapi/linux/vduse.h>
>> +
>> diff --git a/linux-headers/uapi/.gitignore b/linux-headers/uapi/.gitignore
>> new file mode 100644
>> index 0000000000..88829d04e9
>> --- /dev/null
>> +++ b/linux-headers/uapi/.gitignore
>> @@ -0,0 +1,3 @@
>> +**
>> +!**/
>> +!**/*.h
> 
> Nice trick to solve the per patch build issue :-).

:-)

> 

Thanks,
Maxime