From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 81F2945AF0; Wed, 9 Oct 2024 10:58:25 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B867D40E45; Wed, 9 Oct 2024 10:58:20 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id D511840653 for ; Wed, 9 Oct 2024 10:58:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1728464297; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OKwgs+yCkVLm5F4epZfOR+KQLqxnMwZnw42YurDv6mc=; b=Jh1DwNKxchLSzygVhUSWHfuJ3IjGLQzwgDpXNqHFJrOtgl46/WT1F0MJ5MH8FgQBFpDS3Z pzIpCXorDbpd5sPEXy0yavm5I0sgrFTBkz4fYELr28+hevvddq3lPD+G15UGullFCrZGA2 8yZswSaifxMngSCOFBB5QQZ5zIUeIeY= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-608-zkvGcsOGP3mTk8TYqRbQ7g-1; Wed, 09 Oct 2024 04:58:16 -0400 X-MC-Unique: zkvGcsOGP3mTk8TYqRbQ7g-1 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5C0A91955D4E; Wed, 9 Oct 2024 08:58:14 +0000 (UTC) Received: from max-p1.redhat.com (unknown [10.39.208.2]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9DEF2300019E; Wed, 9 Oct 2024 08:58:11 +0000 (UTC) From: Maxime Coquelin To: dev@dpdk.org, techboard@dpdk.org, david.marchand@redhat.com, thomas@monjalon.net, mb@smartsharesystems.com, stephen@networkplumber.org, ferruh.yigit@amd.com Cc: Maxime Coquelin Subject: [PATCH v1 1/3] uapi: introduce kernel uAPI headers import Date: Wed, 9 Oct 2024 10:58:03 +0200 Message-ID: <20241009085805.2638314-2-maxime.coquelin@redhat.com> In-Reply-To: <20241009085805.2638314-1-maxime.coquelin@redhat.com> References: <20241009085805.2638314-1-maxime.coquelin@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch introduces uAPI headers import into the DPDK repository. This import is possible thanks to Linux Kernel licence exception for syscalls: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/LICENSES/exceptions/Linux-syscall-note Header files have to be explicitly imported. Guidelines are provided in the documentation, and helper script is also provided to ensure proper import of the headers (unmodified content from a released Kernel version). Signed-off-by: Maxime Coquelin --- devtools/linux-uapi.sh | 177 +++++++++++++++++++++++++ doc/guides/contributing/index.rst | 1 + doc/guides/contributing/linux_uapi.rst | 70 ++++++++++ kernel/linux/uapi/.gitignore | 4 + kernel/linux/uapi/version | 1 + meson.build | 8 +- 6 files changed, 259 insertions(+), 2 deletions(-) create mode 100755 devtools/linux-uapi.sh create mode 100644 doc/guides/contributing/linux_uapi.rst create mode 100644 kernel/linux/uapi/.gitignore create mode 100644 kernel/linux/uapi/version diff --git a/devtools/linux-uapi.sh b/devtools/linux-uapi.sh new file mode 100755 index 0000000000..175d6a345d --- /dev/null +++ b/devtools/linux-uapi.sh @@ -0,0 +1,177 @@ +#!/bin/sh -e +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2024 Red Hat, Inc. + +# +# Import and check Linux Kernel uAPI headers +# + +base_url="https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/include/uapi/" +base_path="kernel/linux/uapi/" +version="" +file="" +check_headers=0 +errors=0 + +print_usage() +{ + echo "Usage: $(basename $0) [-h] [-i FILE] [-u VERSION] [-c]" + echo "-i FILE import Linux header file. E.g. linux/vfio.h" + echo "-u VERSION update imported list of Linux headers to a given version. E.g. v6.10" + echo "-c check headers are valid" +} + +version_older_than() { + printf '%s\n%s' "$1" "$2" | sort -C -V +} + +download_header() +{ + local header=$1 + local path=$2 + + local url="${base_url}${header}?h=${version}" + + if ! curl -s -f --create-dirs -o $path $url; then + echo "Failed to download $url" + return 1 + fi + + return 0 +} + +update_headers() +{ + local header + local url + local path + + echo "Updating to $version" + for filename in $(find $base_path -name "*.h" -type f); do + header=${filename#$base_path} + download_header $header $filename || return 1 + done + + return 0 +} + +import_header() +{ + local include + local import + local header=$1 + + local path="${base_path}${header}" + + download_header $header $path || return 1 + + for include in $(sed -ne 's/^#include <\(.*\)>$/\1/p' $path); do + if [ ! -f "${base_path}${include}" ]; then + read -p "Import $include (y/n): " import && [ "$import" = 'y' ] || continue + echo "Importing $include for $path" + import_header "$include" || return 1 + fi + done + + return 0 +} + +fixup_includes() +{ + local path=$1 + + sed -i -E -e ' + s/([[:space:](])(__user|__force|__iomem)[[:space:]]/\1/g + s/__attribute_const__([[:space:]]|$)/\1/g + s@^#include @@ + s/(^|[^a-zA-Z0-9])__packed([^a-zA-Z0-9_]|$)/\1__attribute__((packed))\2/g + s/(^|[[:space:](])(inline|asm|volatile)([[:space:](]|$)/\1__\2__\3/g + s@#(ifndef|define|endif[[:space:]]*/[*])[[:space:]]*_UAPI@#\1 @ + ' $path + + # Prepend include path with "uapi/" if the header is imported + for include in $(sed -ne 's/^#include <\(.*\)>$/\1/p' $path); do + if [ -f "${base_path}${include}" ]; then + sed -i "s|${include}|uapi/${include}|g" $path + fi + done +} + +check_header() { + echo -n "Checking $1... " + + if ! diff -q $1 $2 >/dev/null; then + echo "KO" + diff -u $1 $2 + return 1 + else + echo "OK" + fi + + return 0 +} + +while getopts i:u:ch opt ; do + case ${opt} in + i ) file=$OPTARG ;; + u ) version=$OPTARG ;; + c ) check_headers=1 ;; + h ) print_usage ; exit 0 ;; + ? ) print_usage ; exit 1 ;; + esac +done + +shift $(($OPTIND - 1)) +if [ $# -ne 0 ]; then + print_usage + exit 1 +fi + +cd $(dirname $0)/.. + +current_version=$(< ${base_path}/version) + +if [ -n "${version}" ]; then + if version_older_than "$version" "$current_version"; then + echo "Headers already up to date ($current_version >= $version)" + version=$current_version + else + update_headers || exit 1 + fi +else + echo "Version not specified, using current version ($current_version)" + version=$current_version +fi + +if [ -n "${file}" ]; then + import_header $file || exit 1 +fi + +for filename in $(find $base_path -name "*.h" -type f); do + fixup_includes $filename || exit 1 +done + +echo $version > ${base_path}/version + +if [ $check_headers -eq 0 ]; then + exit 0 +fi + +tmpheader="$(mktemp -t dpdk.checkuapi.XXXXXX)" +trap "rm -f '$tmpheader" INT + +echo "Checking imported headers for version ${version}" + +for filename in $(find $base_path -name "*.h" -type f); do + header=${filename#$base_path} + download_header $header $tmpheader || exit 1 + fixup_includes $tmpheader || exit 1 + check_header $filename $tmpheader || errors=$((errors+1)) +done + +echo "$errors error(s) found" + +rm -f $tmpheader +trap - INT + +exit $errors diff --git a/doc/guides/contributing/index.rst b/doc/guides/contributing/index.rst index dcb9b1fbf0..603dc72654 100644 --- a/doc/guides/contributing/index.rst +++ b/doc/guides/contributing/index.rst @@ -19,3 +19,4 @@ Contributor's Guidelines vulnerability stable cheatsheet + linux_uapi diff --git a/doc/guides/contributing/linux_uapi.rst b/doc/guides/contributing/linux_uapi.rst new file mode 100644 index 0000000000..00bc092311 --- /dev/null +++ b/doc/guides/contributing/linux_uapi.rst @@ -0,0 +1,70 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2024 Red Hat, Inc. + +Linux uAPI header files +======================= + +Rationale +--------- + +The system a DPDK library or driver is built on is not necessarily running the +same Kernel version than the system that will run it. +Importing Linux Kernel uAPI headers enable to build features that are not +supported yet by the build system. + +For example, the build system runs upstream Kernel v5.19 and we would like to +build a VDUSE application that will use VDUSE_IOTLB_GET_INFO ioctl() introduced +in Linux Kernel v6.0. + +`Linux Kernel licence exception regarding syscalls +`_ +enable importing unmodified Linux Kernel uAPI header files. + +Importing or updating an uAPI header file +----------------------------------------- + +In order to ensure the imported uAPI headers are both unmodified and from a +released version of the linux Kernel, a helper script is made available and +MUST be used. +Below is an example to import ``linux/vduse.h`` file from Linux ``v6.10``: + +.. code-block:: console + + ./devtools/linux-uapi.sh -i linux/vduse.h -u v6.10 + +Once imported, the header files should be committed without any other change. +Note that all the imported headers will be updated to the requested version. + +Updating imported headers to a newer released version should only be done on +a need basis, it can be achieved using the same script: + +.. code-block:: console + + ./devtools/linux-uapi.sh -u v6.10 + +The commit message should reflect why updating the headers is necessary. + +Once committed, user can check headers are valid by using the same Linux +uAPI script using the check option: + +.. code-block:: console + + ./devtools/linux-uapi.sh -c + +Note that all the linux-uapi.sh script options can be combined. For example: + +.. code-block:: console + + ./devtools/linux-uapi.sh -i linux/virtio_net.h -u v6.10 -c + +Header inclusion into library or driver +--------------------------------------- + +The library or driver willing to make use of imported uAPI headers needs to +explicitly include the header file with ``uapi/`` prefix in C files. +For example to include VDUSE uAPI: + +.. code-block:: c + + #include + diff --git a/kernel/linux/uapi/.gitignore b/kernel/linux/uapi/.gitignore new file mode 100644 index 0000000000..558ba597d6 --- /dev/null +++ b/kernel/linux/uapi/.gitignore @@ -0,0 +1,4 @@ +** +!**/ +!**/*.h +!version diff --git a/kernel/linux/uapi/version b/kernel/linux/uapi/version new file mode 100644 index 0000000000..2b256fd48f --- /dev/null +++ b/kernel/linux/uapi/version @@ -0,0 +1 @@ +v0.0 diff --git a/meson.build b/meson.build index 8b248d4505..85b53cafbb 100644 --- a/meson.build +++ b/meson.build @@ -67,16 +67,20 @@ endif # configure the build, and make sure configs here and in config folder are # able to be included in any file. We also store a global array of include dirs # for passing to pmdinfogen scripts -global_inc = include_directories('.', 'config', +global_inc = [include_directories('.', 'config', 'lib/eal/include', 'lib/eal/@0@/include'.format(host_machine.system()), 'lib/eal/@0@/include'.format(arch_subdir), -) +)] # do configuration and get tool paths subdir('buildtools') subdir('config') +if is_linux + global_inc += include_directories('kernel/linux') +endif + # build libs and drivers subdir('lib') subdir('drivers') -- 2.46.2