From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1C1A242B24; Wed, 17 May 2023 01:06:51 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E9D2E410E4; Wed, 17 May 2023 01:06:50 +0200 (CEST) Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) by mails.dpdk.org (Postfix) with ESMTP id 4E60740EE1 for ; Wed, 17 May 2023 01:06:50 +0200 (CEST) Received: by mail-pg1-f176.google.com with SMTP id 41be03b00d2f7-5304d0d1eddso5043008a12.2 for ; Tue, 16 May 2023 16:06:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=damore-org.20221208.gappssmtp.com; s=20221208; t=1684278409; x=1686870409; h=mime-version:subject:references:in-reply-to:message-id:to:from:date :from:to:cc:subject:date:message-id:reply-to; bh=yLFjtqcCCDS1FXWURaw1lo+n9QpHnKH8Yf62k4UA4xE=; b=p4MLHmT+OJaXRf045CgYv3VYyQ4ReV5rObDbAx2HB989wyFkxPgnnQlwHQMsLRXoCJ e34dgz5elZkonM6aOny3LuThYeD568I2mmkzpTC5S1b8tHEkg66SzlhCMO/muBnLCJYu m4oUIS/tGdwStwv5ZH1d/+BedtUnAnHdgQLLbYSGATTGXd/miNjyPuhxzAU8JMHkICnp YztFRxbXvjkYDXy70diT69wER55rsNStA8TLfOBCv3TlO6tJxj/M69Ap6GXYuk+1+zBG Cz4Vk6epsUhsV7HD1BdiEdVJnBIvfVRfYz4aBW7L8JY71N2HY5ctQA46B6i49lxwjgWa KXIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684278409; x=1686870409; h=mime-version:subject:references:in-reply-to:message-id:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=yLFjtqcCCDS1FXWURaw1lo+n9QpHnKH8Yf62k4UA4xE=; b=FJL3YEVOnDOTXwkT2ipWslS6SJhdrRP/SwzrfixVSGn51UsDRftU0rzql69HAj/zk/ RDFBwux9bcLusPsBwojV923PAINe+4p5ErLJSuuP9nRJaphFwjzyUXhgZmzU6UAD2mb/ DCG9pGH8IHWMTDw/dSJfmPZhhhCHXFBtHzILffKisG6C2kdqhgoC93inOgwrL/mYRzgP cem6xDfIyMmvM0NMfdUW7TOj8/WwCBSimes/jSp46jYsl29w0PsNgOsJDvU4GZMzlqkC JOnOwOUFDRBwPX9PvjB3JViCuXhAwvgJtU1X119dTH13x1/TMgewcwntThXr9w+ZWAiF n/zw== X-Gm-Message-State: AC+VfDxMZsf1agJ+y3Kent/ieEUYxex1prGWceR0dUtTbxc+hno26zn2 bjKf2LfeNYY4DxmSjJNR40RQteU3A6dYb3oYQIM= X-Google-Smtp-Source: ACHHUZ6JaQhhum3N8dVh8maTTveAw+HFpB10dFUNjkVPjXaYqm+Dz6aJ0/prLwOOzaYrDkLE7rWfKw== X-Received: by 2002:a17:902:a602:b0:1ae:691:dfe6 with SMTP id u2-20020a170902a60200b001ae0691dfe6mr10118811plq.64.1684278409078; Tue, 16 May 2023 16:06:49 -0700 (PDT) Received: from [10.41.221.196] ([149.20.194.137]) by smtp.gmail.com with ESMTPSA id kb14-20020a170903338e00b001ae4d4d2676sm385659plb.269.2023.05.16.16.06.48 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 May 2023 16:06:48 -0700 (PDT) Date: Tue, 16 May 2023 16:06:42 -0700 From: Garrett D'Amore To: dev@dpdk.org, Philip Prindeville Message-ID: <470f0311-ed76-4134-8d62-f4db7269f1d6@Spark> In-Reply-To: <60A57C56-D30C-4B0F-BAE9-01BA6FD16E66@redfish-solutions.com> References: <60A57C56-D30C-4B0F-BAE9-01BA6FD16E66@redfish-solutions.com> Subject: Re: DPDK packaging and OpenWrt X-Readdle-Message-ID: 470f0311-ed76-4134-8d62-f4db7269f1d6@Spark MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="64640c87_1de8725a_734c" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --64640c87_1de8725a_734c Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On May 16, 2023 at 12:19 PM -0700, Philip Prindeville , wrote: > Hi, > > I'm a packaging maintainer for some of the OpenWrt ancillary packages, = and I'm looking at upstreaming DPDK support into OpenWrt. Apologies in ad= vance if this is a bit dense (a brain dump) or hops between too many topi= cs. > > I was looking at what's been done to date, and it seems there are a few= shortcomings which I hope to address. > > Amongst them are: > > > I have some related questions. > > 1. OpenWrt already bakes =22vfio.ko=22 and =22vfio-pci.ko=22 here: > > https://github.com/openwrt/openwrt/blob/master/package/kernel/linux/mod= ules/virt.mk=23L77-L117 > > but does so with =22CON=46IG=5FV=46IO=5FPCI=5FIGD=3Dy=22, which seems t= o be specifically for VGA acceleration of guests in a virtualized environ= ment. That seems to be an odd corner case, and unlikely given that OpenWr= t almost always runs on headless hardware. Should this be reverted=3F > > 2. =22vfio.ko=22 is built with =22CON=46IG=5FV=46IO=5FNOIOMMU=3Dy=22, w= hich seems insecure. Can this be reverted=3F > > 3. Is =22uio=5Fpci=5Fgeneric.ko=22 worth the potential insecurity/insta= bility of a misbehaving application=3F My understanding is that it's only= needed on SR-IOV hardware where MSI/MSI-X interrupts aren't supported: i= s there even any current hardware that doesn't support MSI/MSI-X=3F Or am= I misunderstanding the use case=3F *Either* uio=5Fpci=5Fgeneric *or* vfio with NOIOMMU are required for the = vary large number of systems that either lack an IOMMU (btw, that will be= nearly all OpenWRT platforms=21), or elect to run with the iommu unconfi= gured (one justification for doing that - apart from numerous software bu= gs and limitations =E2=80=94 is that the IOMMU can slow down I/O. We actu= ally recommend most of our customers that run dedicated systems run with = the IOMMU disabled for this reason.) vfio with=C2=A0=C2=A0noiommu is preferable. > > 4. Can most functionality be achieved with V=46IO + IOMMU support=3F *If* you have an IOMMU, and you aren=E2=80=99t trying to eke the very las= t bits of performance, yes.=C2=A0=C2=A0But as many systems don=E2=80=99t = have an IOMMU, and as one of the main points of DPDK are extremely perfor= mance sensitive applications, I think the answer is more broadly, =E2=80=9C= no=E2=80=9D. > 11. What is the user-space TCP/IP stack of choice (or reference) for us= e with DPDK=3F IMO, if you=E2=80=99re using DPDK to run *TCP* applications then you=E2=80= =99re probably misusing it =E2=80=94 there isn=E2=80=99t a user land TCP = stack that I would trust.=C2=A0=C2=A0IP/UDP is something we do, and it wo= rks well, but I can tell you already it=E2=80=99s a pain to do *just* IP,= because e.g. routing tables, ARP, etc. all have to be handled. =E2=80=A2 Garrett --64640c87_1de8725a_734c Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline
On May 16, 2023 at 12:19 PM -0700, Philip Prindevil= le <philipp=5Fsubx=40redfish-solutions.com>, wrote:
Hi,

I'm a packaging maintainer for some of the OpenWrt ancillary packages, an= d I'm looking at upstreaming DPDK support into OpenWrt. Apologies in adva= nce if this is a bit dense (a brain dump) or hops between too many topics= .

I was looking at what's been done to date, and it seems there are a few s= hortcomings which I hope to address.

Amongst them are:


I have some related questions.

1. OpenWrt already bakes =22vfio.ko=22 and =22vfio-pci.ko=22 here:

https://github.com/openwrt/openwrt/blob/master/package/kernel/linux/modul= es/virt.mk=23L77-L117

but does so with =22CON=46IG=5FV=46IO=5FPCI=5FIGD=3Dy=22, which seems to = be specifically for VGA acceleration of guests in a virtualized environme= nt. That seems to be an odd corner case, and unlikely given that OpenWrt = almost always runs on headless hardware. Should this be reverted=3F
=
2. =22vfio.ko=22 is built with =22CON=46IG=5FV=46IO=5FNOIOMMU=3Dy=22, whi= ch seems insecure. Can this be reverted=3F

3. Is =22uio=5Fpci=5Fgeneric.ko=22 worth the potential insecurity/instabi= lity of a misbehaving application=3F My understanding is that it's only n= eeded on SR-IOV hardware where MSI/MSI-X interrupts aren't supported: is = there even any current hardware that doesn't support MSI/MSI-X=3F Or am I= misunderstanding the use case=3F

*Either* uio=5Fpci=5Fgeneric *or* vfio with NOIOMMU are required for the = vary large number of systems that either lack an IOMMU (btw, that will be= nearly all OpenWRT platforms=21), or elect to run with the iommu unconfi= gured (one justification for doing that - apart from numerous software bu= gs and limitations =E2=80=94 is that the IOMMU can slow down I/O. We actu= ally recommend most of our customers that run dedicated systems run with = the IOMMU disabled for this reason.)

vfio with&=23160;&=23160;noiommu is preferable.

4. Can most functionality be achieved with V=46IO + IOMMU support=3F

*If* you have an IOMMU, and you aren=E2=80=99t trying to eke the very las= t bits of performance, yes.&=23160;&=23160;But as many systems don=E2=80=99= t have an IOMMU, and as one of the main points of DPDK are extremely perf= ormance sensitive applications, I think the answer is more broadly, =E2=80= =9Cno=E2=80=9D.
11. What is the user-space TCP/IP stack of choice (or reference) for use= with DPDK=3F

IMO, if you=E2=80=99re using DPDK to run *TCP* applications then you=E2=80= =99re probably misusing it =E2=80=94 there isn=E2=80=99t a user land TCP = stack that I would trust.&=23160;&=23160;IP/UDP is something we do, and i= t works well, but I can tell you already it=E2=80=99s a pain to do *just*= IP, because e.g. routing tables, ARP, etc. all have to be handled.&=2316= 0;
  • Garrett
--64640c87_1de8725a_734c--