From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 13F184596E; Thu, 12 Sep 2024 15:48:05 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D61D0427B7; Thu, 12 Sep 2024 15:48:04 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id E27AA42709 for ; Thu, 12 Sep 2024 15:48:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1726148882; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=wOfluECagF2jFahyouKLyO59502m914XQbmekhjo/uE=; b=ADJOyZummLeeECp6kybz2JMsB5jf7+2/PA/lq7+Om5M04Kl9GC9nkyHssbmXOb6rOQ3L1h wNUyCPBFy8lT8uSQE7HFAhLF+hf44gs6J9zZZ8kKXYRXv3J2HK7O3QX6edngICZmlt55Ff 4mha5091JIeO8dwIiu0iG6IcNR9yfLM= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-642-fh9hZFcDMQeAkRXfYRc6LQ-1; Thu, 12 Sep 2024 09:48:01 -0400 X-MC-Unique: fh9hZFcDMQeAkRXfYRc6LQ-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7D5C91977037; Thu, 12 Sep 2024 13:47:59 +0000 (UTC) Received: from [10.39.208.29] (unknown [10.39.208.29]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A0BDA19560AA; Thu, 12 Sep 2024 13:47:56 +0000 (UTC) Message-ID: <234b74ed-5ff7-42fe-a538-81ce91399720@redhat.com> Date: Thu, 12 Sep 2024 15:47:53 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC v3 0/3] Import Kernel uAPI header files To: Ferruh Yigit , david.marchand@redhat.com, thomas@monjalon.net, mb@smartsharesystems.com, stephen@networkplumber.org Cc: dev@dpdk.org, techboard@dpdk.org References: <20240911193224.1966122-1-maxime.coquelin@redhat.com> <6df4dd8b-be68-4ca3-8593-9e694577a4d8@amd.com> <656c4c4b-5aca-4668-b292-a40441fbee17@redhat.com> <974da827-a3a4-4509-a01a-b97356f6af98@amd.com> From: Maxime Coquelin Autocrypt: addr=maxime.coquelin@redhat.com; keydata= xsFNBFOEQQIBEADjNLYZZqghYuWv1nlLisptPJp+TSxE/KuP7x47e1Gr5/oMDJ1OKNG8rlNg kLgBQUki3voWhUbMb69ybqdMUHOl21DGCj0BTU3lXwapYXOAnsh8q6RRM+deUpasyT+Jvf3a gU35dgZcomRh5HPmKMU4KfeA38cVUebsFec1HuJAWzOb/UdtQkYyZR4rbzw8SbsOemtMtwOx YdXodneQD7KuRU9IhJKiEfipwqk2pufm2VSGl570l5ANyWMA/XADNhcEXhpkZ1Iwj3TWO7XR uH4xfvPl8nBsLo/EbEI7fbuUULcAnHfowQslPUm6/yaGv6cT5160SPXT1t8U9QDO6aTSo59N jH519JS8oeKZB1n1eLDslCfBpIpWkW8ZElGkOGWAN0vmpLfdyiqBNNyS3eGAfMkJ6b1A24un /TKc6j2QxM0QK4yZGfAxDxtvDv9LFXec8ENJYsbiR6WHRHq7wXl/n8guyh5AuBNQ3LIK44x0 KjGXP1FJkUhUuruGyZsMrDLBRHYi+hhDAgRjqHgoXi5XGETA1PAiNBNnQwMf5aubt+mE2Q5r qLNTgwSo2dpTU3+mJ3y3KlsIfoaxYI7XNsPRXGnZi4hbxmeb2NSXgdCXhX3nELUNYm4ArKBP LugOIT/zRwk0H0+RVwL2zHdMO1Tht1UOFGfOZpvuBF60jhMzbQARAQABzSxNYXhpbWUgQ29x dWVsaW4gPG1heGltZS5jb3F1ZWxpbkByZWRoYXQuY29tPsLBeAQTAQIAIgUCV3u/5QIbAwYL CQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQyjiNKEaHD4ma2g/+P+Hg9WkONPaY1J4AR7Uf kBneosS4NO3CRy0x4WYmUSLYMLx1I3VH6SVjqZ6uBoYy6Fs6TbF6SHNc7QbB6Qjo3neqnQR1 71Ua1MFvIob8vUEl3jAR/+oaE1UJKrxjWztpppQTukIk4oJOmXbL0nj3d8dA2QgHdTyttZ1H xzZJWWz6vqxCrUqHU7RSH9iWg9R2iuTzii4/vk1oi4Qz7y/q8ONOq6ffOy/t5xSZOMtZCspu Mll2Szzpc/trFO0pLH4LZZfz/nXh2uuUbk8qRIJBIjZH3ZQfACffgfNefLe2PxMqJZ8mFJXc RQO0ONZvwoOoHL6CcnFZp2i0P5ddduzwPdGsPq1bnIXnZqJSl3dUfh3xG5ArkliZ/++zGF1O wvpGvpIuOgLqjyCNNRoR7cP7y8F24gWE/HqJBXs1qzdj/5Hr68NVPV1Tu/l2D1KMOcL5sOrz 2jLXauqDWn1Okk9hkXAP7+0Cmi6QwAPuBT3i6t2e8UdtMtCE4sLesWS/XohnSFFscZR6Vaf3 gKdWiJ/fW64L6b9gjkWtHd4jAJBAIAx1JM6xcA1xMbAFsD8gA2oDBWogHGYcScY/4riDNKXi lw92d6IEHnSf6y7KJCKq8F+Jrj2BwRJiFKTJ6ChbOpyyR6nGTckzsLgday2KxBIyuh4w+hMq TGDSp2rmWGJjASrOwU0EVPSbkwEQAMkaNc084Qvql+XW+wcUIY+Dn9A2D1gMr2BVwdSfVDN7 0ZYxo9PvSkzh6eQmnZNQtl8WSHl3VG3IEDQzsMQ2ftZn2sxjcCadexrQQv3Lu60Tgj7YVYRM H+fLYt9W5YuWduJ+FPLbjIKynBf6JCRMWr75QAOhhhaI0tsie3eDsKQBA0w7WCuPiZiheJaL 4MDe9hcH4rM3ybnRW7K2dLszWNhHVoYSFlZGYh+MGpuODeQKDS035+4H2rEWgg+iaOwqD7bg CQXwTZ1kSrm8NxIRVD3MBtzp9SZdUHLfmBl/tLVwDSZvHZhhvJHC6Lj6VL4jPXF5K2+Nn/Su CQmEBisOmwnXZhhu8ulAZ7S2tcl94DCo60ReheDoPBU8PR2TLg8rS5f9w6mLYarvQWL7cDtT d2eX3Z6TggfNINr/RTFrrAd7NHl5h3OnlXj7PQ1f0kfufduOeCQddJN4gsQfxo/qvWVB7PaE 1WTIggPmWS+Xxijk7xG6x9McTdmGhYaPZBpAxewK8ypl5+yubVsE9yOOhKMVo9DoVCjh5To5 aph7CQWfQsV7cd9PfSJjI2lXI0dhEXhQ7lRCFpf3V3mD6CyrhpcJpV6XVGjxJvGUale7+IOp sQIbPKUHpB2F+ZUPWds9yyVxGwDxD8WLqKKy0WLIjkkSsOb9UBNzgRyzrEC9lgQ/ABEBAAHC wV8EGAECAAkFAlT0m5MCGwwACgkQyjiNKEaHD4nU8hAAtt0xFJAy0sOWqSmyxTc7FUcX+pbD KVyPlpl6urKKMk1XtVMUPuae/+UwvIt0urk1mXi6DnrAN50TmQqvdjcPTQ6uoZ8zjgGeASZg jj0/bJGhgUr9U7oG7Hh2F8vzpOqZrdd65MRkxmc7bWj1k81tOU2woR/Gy8xLzi0k0KUa8ueB iYOcZcIGTcs9CssVwQjYaXRoeT65LJnTxYZif2pfNxfINFzCGw42s3EtZFteczClKcVSJ1+L +QUY/J24x0/ocQX/M1PwtZbB4c/2Pg/t5FS+s6UB1Ce08xsJDcwyOPIH6O3tccZuriHgvqKP yKz/Ble76+NFlTK1mpUlfM7PVhD5XzrDUEHWRTeTJSvJ8TIPL4uyfzhjHhlkCU0mw7Pscyxn DE8G0UYMEaNgaZap8dcGMYH/96EfE5s/nTX0M6MXV0yots7U2BDb4soLCxLOJz4tAFDtNFtA wLBhXRSvWhdBJZiig/9CG3dXmKfi2H+wdUCSvEFHRpgo7GK8/Kh3vGhgKmnnxhl8ACBaGy9n fxjSxjSO6rj4/MeenmlJw1yebzkX8ZmaSi8BHe+n6jTGEFNrbiOdWpJgc5yHIZZnwXaW54QT UhhSjDL1rV2B4F28w30jYmlRmm2RdN7iCZfbyP3dvFQTzQ4ySquuPkIGcOOHrvZzxbRjzMx1 Mwqu3GQ= In-Reply-To: <974da827-a3a4-4509-a01a-b97356f6af98@amd.com> X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 9/12/24 15:16, Ferruh Yigit wrote: > On 9/12/2024 1:08 PM, Maxime Coquelin wrote: >> Hi Ferruh, >> >> On 9/12/24 10:30, Ferruh Yigit wrote: >>> On 9/11/2024 8:32 PM, Maxime Coquelin wrote: >>>> This series enables importing Linux Kernel uAPI headers >>>> into the DPDK repository. It aims at solving alignment >>>> issues between the build system and the system applications >>>> linked ot DPDK libraries are run on. >>>> >>> >>> Hi Maxime, >>> >>> Is the imported Linux Kernel uAPI headers are always backward compatible? >> >> Yes, at least that's what is advertised! :) >> >> " >> The binary interface provided by the kernel to user space cannot be >> broken except under the most severe circumstances. >> " >> > > Ack. > >> >>> If I build and run on identical platform, lets say has kernel v5.4, >>> and DPDK has the kernel uAPI header from v6.10, is this has any chance >>> to create backward compatibility issues? >> >> It should not if backward compatibility is preserved as advertised. >> >>> Or can it enable some features which are indeed not exist/supported in >>> the platform that has old version of kernel? >> >> It can enable new feature, like for example a new ioctl command. >> >> In this case, it should return something like ENOIOCTLCMD if the Kernel >> does not support it. >> > > OK. I think this has the same risk with building dpdk application in a > platform with newer kernel version, and distribute it to the another > platform with older kernel. If interface is backward compatible, this > should be OK. Yes. > > In this approach we are duplicating the kernel headers, which is not a > great option. Also this requires maintenance, updating kernel headers in > DPDK as needed, which is not great in long run. > > I guess intention is to enable a feature that comes with newer kernel > versions, if so why not force enable it in DPDK, like: > #ifndef FEATURE_MACRO > #define FEATURE_MACRO > #define X > #define Y > #define Z > #endif That's exactly what my proposal aims at fixing! We do this a lot in Vhost library, and this is a mess. > If the kernel uAPI is stable, moving these defines to library should be > OK. I see some issues with such solution: 1. This is error prone (e.g. involuntary modifications to redefined code). It can be also quite complex/big to redefine (look at linux/userfaultfd.h) 2. We copy "GPL v2 WITH Linux-syscall-note" definitions into BSD licensed code (but IANAL). > When this feature hits the distribution kernels we can remove > defines from the DPDK library. I think this is easier said than done. Distribution Kernels backport feature on a need basis, it is difficult to ensure all the distribution Kernels have a given feature included. > > OR, won't it help if whoever building DPDK, download kernel headers > package for necessary version, and build DPDK with them? > For this documenting required versions, and a how to documentation to > explain meson commands etc.. may be sufficient. That would add another step to building DPDK. It would need to be a mandatory step, otherwise we would have to continue with all these #ifndef #define.. I think my proposal is the best compromise: 1. No need for extra pre-requisites for building DPDK 2. UAPI included on a need basis (e.g. only vduse.h, only virtio headers, etc..) 3. UAPI headers isolated in a dedicated directory, ensuring no involuntary modifications are done to these files thanks to provided checker that can be integrated into CI.License of the UAPI headers are preseved. 4. Helps with builds reproducibility. > >>> >>>> It can also help simplify spaghetti code done to support >>>> different versions of the Linux Kernel headers used by >>>> the build system. >>>> >>>> Guidelines and import script are also part of first patch. >>>> A uAPI checker script is added in this 3rd RFC, goals is to >>>> use it in CI for any patch touching linux-headers/uapi. >>>> >>>> Follow-up patches are an example of imported uAPI inclusion >>>> of VDUSE header into the Vhost library. >>>> >>>> Morten, I did not (again) apply your Ack on patch 1, as it >>>> has some significant changes and additions. >>>> >>>> Changes in RFC v3: >>>> ================== >>>> - Support nested headers include >>>> - Interactive mode to select which headers to include >>>> - Store version in a file instead of git commit messages >>>> - All imported headers aligned on same version >>>> - Improve loops in scripts (David) >>>> - Update documentation >>>> >>>> >>>> Changes in RFC v2: >>>> ================== >>>> - Fix typos in documentation and commit messags (David, Morten) >>>> - Add uAPI checker script >>>> - Add uAPI to global_inc >>>> - Fix build issues on FreeBSD and documentation (CI, David) >>>> - Simplify import script (David) >>>> >>>> Maxime Coquelin (3): >>>>    uapi: introduce kernel uAPI headers import >>>>    uapi: import VDUSE header >>>>    vduse: use imported VDUSE uAPI header >>>> >>>>   devtools/check-linux-uapi.sh           |  85 ++++++ >>>>   devtools/import-linux-uapi.sh          | 119 +++++++++ >>>>   doc/guides/contributing/index.rst      |   1 + >>>>   doc/guides/contributing/linux_uapi.rst |  71 +++++ >>>>   lib/vhost/meson.build                  |   5 +- >>>>   lib/vhost/vduse.c                      |   2 +- >>>>   lib/vhost/vduse.h                      |  22 -- >>>>   linux-headers/uapi/.gitignore          |   4 + >>>>   linux-headers/uapi/linux/vduse.h       | 353 +++++++++++++++++++++++++ >>>>   linux-headers/uapi/version             |   1 + >>>>   meson.build                            |   8 +- >>>>   11 files changed, 642 insertions(+), 29 deletions(-) >>>>   create mode 100755 devtools/check-linux-uapi.sh >>>>   create mode 100755 devtools/import-linux-uapi.sh >>>>   create mode 100644 doc/guides/contributing/linux_uapi.rst >>>>   create mode 100644 linux-headers/uapi/.gitignore >>>>   create mode 100644 linux-headers/uapi/linux/vduse.h >>>>   create mode 100644 linux-headers/uapi/version >>>> >>> >> >