From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D99EFA034C for ; Thu, 28 Jul 2022 23:06:28 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 66DBA40151; Thu, 28 Jul 2022 23:06:28 +0200 (CEST) Received: from sonic309-21.consmr.mail.ne1.yahoo.com (sonic309-21.consmr.mail.ne1.yahoo.com [66.163.184.147]) by mails.dpdk.org (Postfix) with ESMTP id F0D2F4014F for ; Thu, 28 Jul 2022 23:06:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1659042386; bh=bjgWWpFJ8jc6c7O0cxb4QXTGo6UpQdGBwaZlT+7sw2s=; h=From:Subject:Date:References:Cc:In-Reply-To:To:From:Subject:Reply-To; b=SpDaN7DlIvQil55VvFJSc6VASpBIIu7YYqsltluVWpsBkVUHQloDlYuob9pDAoCLrr12pbF2zEuqjCibW7FcZbp3WI7W8+Je5+WmbbeX/JOvNhLk6iSBGZVbN0xSfynOHxz35+OyA1Vw8Vner1jXwbG22/UOoFpYoPGYUlkzVl/KR7nZT+yEGtAQ4TMO1BbyTI/eCC2W95mDH2Q5782dlfHrzqpU8GoYriIfTasOdd9KOfLJdvgTHr3oUW6OPX9dnF8HPtL07OmMxCU6vKTblQpsrPoHaEQXMZA2R9TBWduLJaarCjtlG3yCGXCpioNFm4VUinCiee0xDOFZyPSzBw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1659042386; bh=Bb6TIkpJynOWhMHr2G0mLrxpTn1RTBneH+gJXs2rpae=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=gfgK9E5zMKdprZY+6Ud18SdIFnUdeD0MxTuyjJjLiDxG/zeJU+4upcQtN515ItQthlouw14G4FnccLNu2QGX8qAsUpDixhnHppW90PIsF7VeiINdzipYKQiuzTTmwt6vNiB4eDwuwKpzq1YGNa0P2hLxSlRNvE48bL9K2OW+7u4l9KvXCgJ3K+msENvYZtQiRsw0PWlnf+3RKYN4QPpufwIDp2+6gQj+8PkyZZIH6nuCKQLZKAhFmxD4P2pRmNWYiuoA7cA8Xoe6zUt/lxj9KbsdzmARv/H19zwyd71JRbZsd/rI5TOCW8OcrzVZZjE5GkWqWM1xnlX/YEmtCpgjPQ== X-YMail-OSG: caqQMY8VM1lObzB1XIzga2HMWjn8smIRA1JB9e011U0Z4Ream4jVJ3lXaltzQoo Nh8kYkDBCdgr5Ur4gTp.Ac0qFa1S3cebz0ncRNnua81l9nQeuTt3MWydZ9oKKggAr2H.j_oDjsFx nYzRL0OrhWTGxegZ7WJ8I78H_uqC309UzPkadBSTbY3Ef9d4kHkMt2CIf44_spG3QHlGiqWHHU4e swcma9PNLsmIs7AqrLPad34p1NKsR.EGurdV5L9vLXZHLebX6yQi1AryuCfKe6lfVeQ6wU0OOMW2 Q7.KWWQLx6gYfagXnCSc2FQjkQpItt6yCBukbbs3kiXEJJSio6Ubp98RKbG2YkOn.WYFVt5PZD.z u35g7aFKz3wMBScU9ADrsizbZ0aH0IPBVNnoLcyDg0s0EqEaGSkGn4YsGjKUU3tITThJu.Kv_v3l X0GyiC7ePZqJDB85cGzyXS3wxEPc5_h0mmIWTa4oKhK9aKSTHSP1IHw6j_JEXUL8HjjmvUipLdif BdwNM1sM5nj_FXGZ4hIIj15JvVD1I41koGNkDRS0_d8pun6N7xBJWzXRBkvZ.jHycPK7alYHvGJF ITqqPXxo9sy8ldHIgxn3MpvB.CFQjpgRAdButkhaHibnWAWEtDEHxZIJ6o1KU1T8ygaYZkgOULvU bnzMjhNP4wpqHyrXvx70MyUP2wZCFI54uOOhO.65Mwx5EbXOwsaytllXsL8ypDneniik53dzC2Io EnquFj9ABlriPcGMGxIfC2JSa479K3K_CBfnJcZGzwsMjtnFTGiI8RzPU1ZxDZiaKZi_zANysxZn t8FxjpAfTbBaiAZo9QJnMuujElHGMHUorLraOc8z33n0gGgMkf8mtMYBhAmtPNTJqtjNcQZ3K4CM e6U38LmUiVSnHmSSrlMJP.uHIQpLihA52DPfnXQB8RAMxk2Gp7FJPGHeXyhNqKwuDr_0fQ38xbKC .9iQJQI.nzFKP4wuqDkkuYIHjH3F4liwJdQ2Kzbru3b5o4EbxIIdGt3kPB4Zny7QnRCYRQfD6hNy FzHzBoV_Ik8zi2hzSrWFpmAGRo7yCoN.cbXan27UnPSORC8D38DnzplKWWG4zwFB_.ZwofZWusOP T.x5cIAuaf2QiKaNPN1KxyVoWkCNVhBGE1T23WZiYXySwMTON9gJErrLJueYTuIOQ1xo1ym39.9G r.E9rtdyFPdywVKbA0bMAu8mOQBY.SDbciSmGet4Qjo1bzy9WCloag_YIX2J_MwL0ymYgucbaZpk sEarubEhd.ciDB5kWft1MokC7VjNBJQ1eabmCICzUqnSRoxHL6OnAYYU6T9t9eqXBmisTsNLmFoU BMSlAuOcoH5BJCWr.Srkc.d0mxV06D1q73_S_c0K8yQq2DN4NKUoQ_smE.xiijZdL.MX6aQdNPZS Ek7v7bpPFwlltlcau_ketkQ2Ed8jaJrhO_7JDr4ZxFBE2fVtxJymPOLgiH.9CtrGoFmtTgvAMR6m 9E3U9RycG_lJKunGTSB3elFtUmoccUOfud_xrZG90Hb41_2hCx9x4Jp3XEOL7JNPGuoaM64zajrU qJs5reobXwnYQ4Fs4_J0qETkdjl3.AwnmhQCqJ2b.cmkQme_hQY9ZbTW.dn873b7sPGLGHWZpvsr YLrwPP.fbGn6EzMKq_hXmsWQQjRSSpv_P_JAFktaKGmbZ2h9yvpGpaf2Gc1OEO1S4Fpy0.VRd0QA Zm0wtk2vyHJybBTQXiOe8Q_MREhurKdfYaKTV.w1S8i4Uo_OBEF_87up4lWMnnRWojVJfGME5aB1 9OyjcNDdQcMqjmOutYgw8A078qI_vTFK1AggZH4jJUR69bB8ybQ4Yo5sw7zWHXmoeVRa.EDScemQ 088cv1KyMA8YbXc8VJcyClZ_5H3HKroPnQklRDJ_l9G2WjitT9OpRnoZjyO2IbaSIomZBwIOC_w0 XqgONe4up2j.hJ3NM2phGHNb02IbW0vu.Ft2oe6.mHo0U.wn7QV7uatjrUTT90PNfjxkv2mrqcAK 4HpoCopQIRCgwBA_HoJyYTFdoqqY10xeNLQzIEdoKj1jULG9pnhDf8mZ9fy0rueaAZ2beZKbKA1Z qk_qv.JeUnRn8HaP7KvulmGD_TEl2bR.4qfE8pkyfspUJY2eZ8o228Z02JYzwZlrDXpbw463Z4vI fapihRVOnmBXqrS8AtREtrylp2hqxBx4aT4QEoVUIO24sndIWtsDxDmBP_3R5rup9JblW_6lqYov UWC4QuFfdExgfJj_YcUMdLj93mjkolTQg9xOhwviHSmkS_G7WCoT1oDw- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.ne1.yahoo.com with HTTP; Thu, 28 Jul 2022 21:06:26 +0000 Received: by hermes--production-ne1-74ddcb6b46-t2ktp (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID d6f570255ac9992c14a4fb63d2257045; Thu, 28 Jul 2022 21:06:21 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable From: IraM Mime-Version: 1.0 (1.0) Subject: Re: OpenWRT Related Question Date: Thu, 28 Jul 2022 17:06:18 -0400 Message-Id: References: <2285895.BjyWNHgNrj@thomas> Cc: John McNamara In-Reply-To: <2285895.BjyWNHgNrj@thomas> To: Thomas Monjalon , users@dpdk.org X-Mailer: iPhone Mail (19F77) X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Hi all I haven=E2=80=99t been successful in getting a DPDK app running on OpenWRT. running into some issues. Hope to get some help here. I am starting with the basics. =20 - Followed https://doc.dpdk.org/guides/howto/openwrt.html and https://git= hub.com/k13132/openwrt-dpdk - Ubuntu 20.04 as build environment. OpenWRT (target is x86_64) is getti= ng cross compiled on Ubuntu. - OpenWRT is at version 22.03, DPDK is 20.11 OpenWRT image that is build - gets populated with DPDK libraries and tools - Boots up on a VM just fine I couldn't see a way to use mason for building the samples during the build s= o I created an OpenWRT manifest for building it and including it in the imag= e. My manifest makes l2fwd show up as a package during "make menuconfig" and th= e code gets compiled and copied to the image. Having said this, the main issue is that l2fwd starts up but EAL complains t= hat no ports being found.=20 Here are some info from the OpenWRT VM: Dpdk-devbind.py -s shows: Network devices using DPDK-compatible driver =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D 0000:03:00.0 'VMXNET3 Ethernet Controller 07b0' if=3Deth0 drv=3D uio_pci_gen= eric unused=3Dvfio-pci 0000:0b:00.0 'VMXNET3 Ethernet Controller 07b0' if=3Deth0 drv=3D uio_pci_gen= eric unused=3Dvfio-pci Network devices using kernel driver =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D 0000:13:00.0 'VMXNET3 Ethernet Controller 07b0' if=3Deth0 drv=3Dvmxnet3 unus= ed=3Dvfio-pci,uio_pci_generic L2fwd gives the following output: root@OpenWrt:/# l2fwd -- -p 3 EAL: Detected CPU lcores: 6 EAL Detected NUMA nodes: 1 EAL: Detected static linkage of DPDK EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'PA' EAL: VFIO support initialized MAC updating enabled EAL: Error - exiting with code 1 Cause: No Ethernet ports - bye Best I can tell is l2fwd is looking for vfio drivers, where only uio drivers= are loaded. I tried binding VFIO drivers using dpdk-devbind.py --bind=3Dvfio-pci 0000:03:00.0=20 But it fails as follows: Vfio-pci: probe of 0000:0b:00.0 failed with error -22 Vfio-pci: probe of 0000:0b:00.0 failed with error -22 Lspci: unable to load libkmod resources: error -12 Error: bind failed for 0000:0b:00.0 - Cannot bind to driver vfio-pci: [Errno= 19] No such device Error: unbind failed for 0000:0b:00.0 - Cannot open /sysy/bus/pci/drivers//u= nbind: [Errno 13] Permission denied: '/sys/bus/pci/drivers//unbind' So.. although the following kernel options are enabled, per https://doc.dpdk= .org/guides/howto/openwrt.html CONFIG_VFIO_IOMMU_TYPE1=3Dy CONFIG_VFIO_VIRQFD=3Dy CONFIG_VFIO=3Dy CONFIG_VFIO_NOIOMMU=3Dy CONFIG_VFIO_PCI=3Dy CONFIG_VFIO_PCI_MMAP=3Dy CONFIG_HUGETLBFS=3Dy CONFIG_HUGETLB_PAGE=3Dy CONFIG_PROC_PAGE_MONITOR=3Dy I can't load a VFIO driver on OpenWRT. UIO driver loads fine, but l2fwd isn't finding the UIO ports. As a side experiment, I built l2fwd on my Ubuntu build machine without cross= compiling for OpenWRT, and copied it over to my OpenWRT VM. Interestingly,= that exe runs find and finds the UIO drivers. Sent from my iPhone > On Jul 21, 2022, at 03:26, Thomas Monjalon wrote: >=20 > =EF=BB=BF21/07/2022 03:08, R T: >>> On Wednesday, July 20, 2022 at 06:03:32 AM EDT, Thomas Monjalon wrote: =20 >>>=20 >>> We have this howto page about OpenWRT: >>> https://doc.dpdk.org/guides/howto/openwrt.html >>> I am not sure how much it is up to date, >>> please do not hesitate to give feedback if anything can be improved. >>=20 >> One of the items that I'm not seeing, where the regular mason build env o= n Ubuntu was pretty clear on, is building any of the example for testing on O= penWRT. >> How would I go about building and installing one or more of the DPDK exam= ples on the OpenWRT Image? >=20 > It should not be different of any other OS. > You can compile examples either by adding -Dexamples=3Dall in meson comman= d, > or by calling make in the example directory. >=20 >=20