From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6548942B29; Tue, 16 May 2023 22:43:28 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 013CA400D6; Tue, 16 May 2023 22:43:28 +0200 (CEST) Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by mails.dpdk.org (Postfix) with ESMTP id 898634003C for ; Tue, 16 May 2023 22:43:27 +0200 (CEST) Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-1ae3f6e5d70so1026165ad.1 for ; Tue, 16 May 2023 13:43:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20221208.gappssmtp.com; s=20221208; t=1684269806; x=1686861806; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=sGsqttpBfO61eDgwymPKEw/2r3O9Nc9nZ9Vw5bKzTd8=; b=Pe9CNVMDHoXMJ1PiTF8g4YgY4Hj2ontNv33Jybz9dueoI72B+6rqlP9eYW/v49kKoR OvK264WTn8rA3DVUg+rXaJ+3B4s9Y9lRpPhFgu7cT6OZm+BopCnNqsgmx/f95b+DPfw2 Dk9NoKCVPrB4nyyGMgUP8KvRBxARUJddX+n8efdxc0Oe4F6C6BCdGhwN4Q6un8QGUKsU Rm/XJPEOjZ9iq7YjmU4FaOo+tTzhRE5nMSvAOwAbKBovcnDSj4RMLd2nUgu1+srxk1+i 5uiCtunMVLdS0ekM2TZo/n0Xwpqy2YWjtzQwHBNbStTv6jEnjjNWWA+xvDPZT2qq6nvx qQ9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684269806; x=1686861806; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sGsqttpBfO61eDgwymPKEw/2r3O9Nc9nZ9Vw5bKzTd8=; b=Ig+duFNqjAuKCtcMwsuVgPc09WJiwV1w3nBKdnUSkUZl1mCN+OVgEkFsi6GRmYAH6/ 8NrjehLpSAkNk8PCoFRFCRmcNQIUpJpANiDSCn8l91DC/18wqEwUTwOf9YDaCD6qv+Nq Sm2RM/gCYWnYmsb847zoJMMf9S9Cwy3qnuqHcmD9FR9nL/4hoGcVHqCokGL2zxTagOOr PbRxl12WIQ9R6oPxLXcgaiGhvhBz1jWgVWEpDcs5WcOBVrv5zIopLYsJtH++9GeKkUEz 1ED5EffW7fuUvnGzlSbjta7P0VNsudChOABczgp48B0dyQKtA9d/Bn2Z9HRNZGdFgROn GlrQ== X-Gm-Message-State: AC+VfDzmI2N0z3XTcSxuUSBo2DOQ5mtt4Sh9m3hq6nHdT8fu/hCPscnW dLOFawBTkaQyPIpo7vAfUSPNNrgpr+/Ami/Pdvw3Bw== X-Google-Smtp-Source: ACHHUZ5/q84h9MjoUzWhHO25WQ/tSBcYuJhDqHQvM9zZZKRxbVcbpbycgMvg63dtxeEOLh7eQP6bjw== X-Received: by 2002:a17:902:db10:b0:1ac:5c90:23e with SMTP id m16-20020a170902db1000b001ac5c90023emr46658596plx.7.1684269806406; Tue, 16 May 2023 13:43:26 -0700 (PDT) Received: from hermes.local (204-195-120-218.wavecable.com. [204.195.120.218]) by smtp.gmail.com with ESMTPSA id t12-20020a170902e84c00b001a687c505e9sm15973079plg.237.2023.05.16.13.43.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 13:43:25 -0700 (PDT) Date: Tue, 16 May 2023 13:43:23 -0700 From: Stephen Hemminger To: Philip Prindeville Cc: dev@dpdk.org Subject: Re: DPDK packaging and OpenWrt Message-ID: <20230516134323.4e2d5db9@hermes.local> In-Reply-To: <60A57C56-D30C-4B0F-BAE9-01BA6FD16E66@redfish-solutions.com> References: <60A57C56-D30C-4B0F-BAE9-01BA6FD16E66@redfish-solutions.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Tue, 16 May 2023 13:18:40 -0600 Philip Prindeville wrote: > Hi, > > I'm a packaging maintainer for some of the OpenWrt ancillary packages, and I'm looking at upstreaming DPDK support into OpenWrt. Apologies in advance if this is a bit dense (a brain dump) or hops between too many topics. > > I was looking at what's been done to date, and it seems there are a few shortcomings which I hope to address. > > Amongst them are: > > * getting DPDK support into OpenWrt's main repo for the kmod's and into the packages repo for the user-space support; DPDK kernel modules are deprecated, creating more usage of them is problematic. > > * making DPDK supported on all available architectures (i.e. agnostic, not just x86_64 specific); > > * integrating into the OpenWrt "make menuconfig" system, so that editing packages directly isn't required; Does openwrt build system support meson? > * supporting cross-building and avoiding the [flawed] assumption that the micro-architecture of the build server is in any way related to the processor type of the target machine(s); > > * integration into the OpenWrt CI/CD tests; > > * making the kernel support as secure/robust as possible (i.e. avoiding an ill-behaved application taking down the kernel, since this is a firewall after all); Not a problem with vfio > > * avoiding conflict with other existing module or package functionality; > > * avoiding, to the extent possible, introducing one-off toolchain dependencies that unnecessarily complicate the build ecosystem; > > To this end, I'm asking the mailing list for guidance on the best packaging practices that satisfy these goals. I'm willing to do whatever heavy lifting that's within my ability. > > I have some related questions. > > 1. OpenWrt already bakes "vfio.ko" and "vfio-pci.ko" here: > > https://github.com/openwrt/openwrt/blob/master/package/kernel/linux/modules/virt.mk#L77-L117 > > but does so with "CONFIG_VFIO_PCI_IGD=y", which seems to be specifically for VGA acceleration of guests in a virtualized environment. That seems to be an odd corner case, and unlikely given that OpenWrt almost always runs on headless hardware. Should this be reverted? Yes. > > 2. "vfio.ko" is built with "CONFIG_VFIO_NOIOMMU=y", which seems insecure. Can this be reverted? No. Most of OpenWrt's systems do not have an IOMMU. > 3. Is "uio_pci_generic.ko" worth the potential insecurity/instability of a misbehaving application? My understanding is that it's only needed on SR-IOV hardware where MSI/MSI-X interrupts aren't supported: is there even any current hardware that doesn't support MSI/MSI-X? Or am I misunderstanding the use case? Use VFIO noiommu, it is more supported and tested upstream. With PCI generic, no interrupts work. > 4. Can most functionality be achieved with VFIO + IOMMU support? Yes. > 5. I've seen packaging for the "iommu_v2.ko" module done here: > > https://github.com/k13132/openwrt-dpdk/blob/main/packages/kmod-iommu_v2/Makefile#L22-L42 > > Is this potentially useful? What platforms/architectures use this driver? > > 6. Hand editing a wrapper for dpdk.tar.gz is a non-starter. I'd rather add Kconfig adornments to OpenWrt packaging for the wrapper so that options for "-Denable_drivers=" and "-Dcpu_instruction_set=" can be passed in once global build options for OpenWrt have been selected. Defaulting the instruction set to the build host is going to be wrong at least some of the time, if not most of the time. For x86_64, what is a decent compromise for a micro-architecture that will build and run on most AMD and Intel hardware? What's a decent baseline for an ARM64 micro-architecture that will build and run on most ARM hardware? > > 7. Many embedded systems don't build with glibc because it's too bloated (and because critical fixes sometimes take too long to roll out), and instead use MUSL, eglibc, or uClibc (although the last one seems to be waning). Only glibc supports from what I can tell. Can broader support for other C runtimes be added? Can RTE_BACKTRACE be made a parameter or conditionally defined based on the runtime headers? (autotools would be and HAVE_EXECINFO_H would be really handy here, but I'm not sure how to make this play well with meson/ninja, and truth be told I'm an old school Makefile + autotools knuckle-dragger). You could do without backtrace, but then if DPDK applications you are flying blind. > 8. How do I validate that DPDK is properly being built with the cross-tools and not native tools? Even when building x86_64 targets on an x86_64 build host, we end up using a custom toolchain and not the "native" compiler that comes with the distro. > > 9. What is the parity between AMD64 and ARM64? Do both platforms offer equal functionality and security, if not performance? Apples/Oranges. > 10. Who else is using DPDK with OpenWrt that is open to collaboration? > > 11. What is the user-space TCP/IP stack of choice (or reference) for use with DPDK? No user-space TCP/IP stack is really robust and that great. VPP has one but it is likely to be specific to using VPP and not sure if you want to go there. I don't think Fedora/Suse/Debian/Ubuntu have packaged any userspace TCP yet.