From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A65DDA0032; Thu, 12 May 2022 11:46:12 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4D58940E64; Thu, 12 May 2022 11:46:12 +0200 (CEST) Received: from smtp-relay-internal-0.canonical.com (smtp-relay-internal-0.canonical.com [185.125.188.122]) by mails.dpdk.org (Postfix) with ESMTP id 324FD4014F for ; Thu, 12 May 2022 11:46:11 +0200 (CEST) Received: from mail-ej1-f71.google.com (mail-ej1-f71.google.com [209.85.218.71]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id C69EA3F32E for ; Thu, 12 May 2022 09:46:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1652348770; bh=rXjl/c92zJVG+RD/jtEYeA57XaOOVNaVZHZOS6YwSs0=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=Os1lSUemVrJJfU2OL5x+2aVRib/tRx+JSJi4m7bXjeGIM7gQplbVI+tr79qadmRsP 5ZfNeOEJTOBMqZKTLkof2LZnhc5md+3x27FotG8b7kYJhwv9wtNTEuxIpRZP19gAkD cSGePinU702r9Wtb76O1m52IeFIps9s6Ut63/zXgaybPZ68QnA3g699ktufs0KJT8u UQmYu6Ox8WnyPoJB4HpiSxEmNa5XRMTLu1AhY671vfTP8sIULY2comJ09iBBZrpC75 GVjDrFK367vAyhzxvrPY27dbnO54H39GG4wj05FjTWJH7eAIknH8zBfuQ19b5/m4nd qOiddjn3/BBEg== Received: by mail-ej1-f71.google.com with SMTP id nb10-20020a1709071c8a00b006e8f89863ceso2568818ejc.18 for ; Thu, 12 May 2022 02:46:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=rXjl/c92zJVG+RD/jtEYeA57XaOOVNaVZHZOS6YwSs0=; b=PbpPYxUZqCUJACgE7m5A1xWdULsbm0Pd+7eX+Y/ma9A9eZux6IJkt/Nn/RLSN3j9DL G+4PgLEEd74/VI5aIDK8twam/Aurg+1N+XKNBTVDxq0xMDaW1ZG3Hvm++7uAWSiX9LDB 0mZMmDhd0AQAss6UlNs7t5qFtfTpDJ9GL4gqNKDzyx2qjs/iAlxRe/oErIki1MbqCGeT Hp3O5mzP2eXYI7tRkEa2/pfciGCoZjVACMJWqVtOs6+9kxdd/FlndchfA3hyotwmGIGV o6oFQIk66aJ8adQyoyKLud56ex6xVBGx7ubI/UdhEL3ns1+OMsshg188Yo4GrOpZ/S+y Yf7A== X-Gm-Message-State: AOAM531rsPEcJpsS9AhzQBGHTjoUrhcZeMbIaE1/vOlv/+1x+p5yMFLM rlEbgiTgUH/8S+xekD9P8WDfARslFDjBj+tGIXeMiHNdg8eafnHgeh9O2LtpUMFxFARJRz1ZswL 80dy+wNd2Ld/o07oo+Fhu X-Received: by 2002:a17:906:9b96:b0:6f5:cba:b665 with SMTP id dd22-20020a1709069b9600b006f50cbab665mr29015305ejc.505.1652348770372; Thu, 12 May 2022 02:46:10 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx+GHhFh3z6TznBSku6bmbYYg3hs46FmqqpfZrgiGlYadFyN1832RYg3tizOEPnMiy89np1+Q== X-Received: by 2002:a17:906:9b96:b0:6f5:cba:b665 with SMTP id dd22-20020a1709069b9600b006f50cbab665mr29015277ejc.505.1652348769973; Thu, 12 May 2022 02:46:09 -0700 (PDT) Received: from [192.168.123.55] (ip-062-143-094-109.um16.pools.vodafone-ip.de. [62.143.94.109]) by smtp.gmail.com with ESMTPSA id y20-20020aa7d514000000b0042617ba63basm2286759edq.68.2022.05.12.02.46.09 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 12 May 2022 02:46:09 -0700 (PDT) Message-ID: <9f966213-241b-20be-60f3-bd5ec456d051@canonical.com> Date: Thu, 12 May 2022 11:46:08 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.8.0 Subject: Re: [PATCH 00/11] Introduce support for RISC-V architecture Content-Language: en-US To: =?UTF-8?Q?Stanis=c5=82aw_Kardach?= Cc: Frank Zhao , Sam Grove , Marcin Wojtas , upstream@semihalf.com, dev References: <20220505173003.3242618-1-kda@semihalf.com> <07665f47-a437-d1df-d613-e01062a6668c@canonical.com> From: Heinrich Schuchardt In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 5/12/22 10:35, Stanisław Kardach wrote: > On Thu, May 12, 2022 at 10:04 AM Heinrich Schuchardt > wrote: >> >> On 5/5/22 19:29, Stanislaw Kardach wrote: >>> This patchset adds support for building and running DPDK on 64bit RISC-V >>> architecture. The initial support targets rv64gc (rv64imafdc) ISA and >>> was tested on SiFive Unmatched development board with the Freedom U740 >>> SoC running Linux (freedom-u-sdk based kernel). >>> I have tested this codebase using DPDK unit and perf tests as well as >>> test-pmd, l2fwd and l3fwd examples. >>> The NIC attached to the DUT was Intel X520-DA2 which uses ixgbe PMD. >>> On the UIO side, since U740 does not have an IOMMU, I've used igb_uio, >>> uio_pci_generic and vfio-pci noiommu drivers. >>> >>> Commits 1-2 fix small issues which are encountered if a given platform >>> does not support any vector operations (which is the case with U740). >>> Commit 3 introduces EAL and build system support for RISC-V architecture >>> as well as documentation updates. >>> Commits 4-7 add missing defines and stubs to enable RISC-V operation in >>> non-EAL parts. >>> Commit 8 adds RISC-V specific cpuflags test. >>> Commit 9 works around a bug in the current GCC in test_ring compiled >>> with -O0 or -Og. >>> Commit 10 adds RISC-V testing to test-meson-builds.sh automatically >>> iterating over cross-compile config files (currently present for >>> generic rv64gc and SiFive U740). >>> Commit 11 extends hash r/w perf test by displaying both HTM and non-HTM >>> measurements. This is an extraneous commit which is not directly >>> needed for RISC-V support but was noticed when we have started >>> gathering test results. If needed, I can submit it separately. >>> >>> I appreciate Your comments and feedback. >>> >>> Best Regards, >>> Stanislaw Kardach >> >> On an SiFive Unmatched board with Ubuntu Jammy I have been compiling >> DPDK origin/main (36db4a1ad464) with this series and the test patches >> that you split off. > Thank you for testing! >> >> The fast tests are either skipped or succeed. But I have seen errors for >> the performance tests: >> >> 4 1GiB hugepages defined >> no network devices bound >> vfio_pci loaded (no IOMMU) >> >> sudo meson -t 30 >> >> 129/173 DPDK:perf-tests / pmd_perf_autotest FAIL 7.85s (exit status 255 >> or signal 127 SIGinvalid) >> >>> DPDK_TEST=pmd_perf_autotest MALLOC_PERTURB_=135 >> build/app/test/dpdk-test > This test I believe requires at least a single NIC port. Conversely > fast-tests seem to fail if a device is bound. DPDK CI doesn't bind any > on fast-tests, I'm not sure if perf-tests suite is run there. > On my local x86 machine perf-tests suite fails when no device is bound too. > >> 136/173 DPDK:perf-tests / ipsec_perf_autotest FAIL 4.47s (exit status >> 255 or signal 127 SIGinvalid) >> >>> MALLOC_PERTURB_=181 DPDK_TEST=ipsec_perf_autotest >> build/app/test/dpdk-test > On my unmatched I don't have openssl compiled so most crypto tests are > skipped (since no openssl is installed), aside of this one and it > indeed fails. I'm not sure whether it's a matter of a lack of a guard > to skip this test if there's no crypto backend built or if it's a real > issue. > I admit I haven't tried crypto libraries with RISC-V openssl as > there's no real HW offloading on Unmatched. > >> 160/173 DPDK:driver-tests / link_bonding_mode4_autotest FAIL 31.80s >> killed by signal 11 SIGSEGV >> >>> MALLOC_PERTURB_=62 DPDK_TEST=link_bonding_mode4_autotest >> build/app/test/dpdk-test > I use -t40 when testing on Unmatched. Unfortunately lots of the tests > hit timeouts. > >> >> These are the patches used: >> >> # [PATCH 1/1] lpm: add a scalar version of lookupx4 function >> https://inbox.dpdk.org/dev/20220510115824.457885-1-kda@semihalf.com/ >> # [PATCH 1/1] examples/l3fwd: fix scalar LPM compilation >> https://inbox.dpdk.org/dev/20220510115844.458009-1-kda@semihalf.com/ >> # [PATCH 1/1] test/hash: report non HTM numbers for single r/w >> https://inbox.dpdk.org/dev/20220510115734.457718-1-kda@semihalf.com/ >> # [PATCH 1/1] test/ring: remove excessive inlining >> https://inbox.dpdk.org/dev/20220510115758.457794-1-kda@semihalf.com/ >> # [PATCH v3 0/8] Introduce support for RISC-V architecture >> https://inbox.dpdk.org/dev/20220510154849.530872-1-kda@semihalf.com/ >> # [PATCH 1/1] drivers: define OPENSSL_API_COMPAT >> >> https://inbox.dpdk.org/dev/20220510150635.61975-1-heinrich.schuchardt@canonical.com/ >> >> Running sudo app/test/dpdk-test does not show the errors: >> >> RTE>>pmd_perf_autotest >> Start PMD RXTX cycles cost test. >> At least 1 port(s) used for perf. test >> Test Failed >> RTE>>ipsec_perf_autotest >> USER1: rte_ipsec_pkt_crypto_prepare fail >> Test Failed >> RTE>>link_bonding_mode4_autotest >> ... >> Test OK >> RTE> >> >> Best regards >> >> Heinrich+ The above failures for performance tests without a bound NIC can be reproduced on x86_64. So this is not RISC-V specific. With both ports of an Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 bound to the sfio-pci driver some fast tests fail on the Unmatched board: 16/173 DPDK:fast-tests / eal_flags_n_opt_autotest FAIL 5.54s (exit status 255 or signal 127 SIGinvalid) >>> DPDK_TEST=eal_flags_n_opt_autotest MALLOC_PERTURB_=31 build/app/test/dpdk-test --file-prefix=eal_flags_n_opt_autotest 18/173 DPDK:fast-tests / eal_flags_no_huge_autotest FAIL 5.31s (exit status 255 or signal 127 SIGinvalid) >>> DPDK_TEST=eal_flags_no_huge_autotest MALLOC_PERTURB_=150 build/app/test/dpdk-test --file-prefix=eal_flags_no_huge_autotest 21/173 DPDK:fast-tests / eal_flags_vdev_opt_autotest FAIL 5.38s (exit status 255 or signal 127 SIGinvalid) >>> MALLOC_PERTURB_=71 DPDK_TEST=eal_flags_vdev_opt_autotest build/app/test/dpdk-test --file-prefix=eal_flags_vdev_opt_autotest 25/173 DPDK:fast-tests / eal_flags_misc_autotest FAIL 5.58s (exit status 255 or signal 127 SIGinvalid) >>> DPDK_TEST=eal_flags_misc_autotest MALLOC_PERTURB_=130 build/app/test/dpdk-test --file-prefix=eal_flags_misc_autotest The eal* tests succeed on x86_64 with a bound Intel I211 NIC. If the eal* tests are run depends on the installed Linux packages. I have used 'apt build-dep dpdk' on Ubuntu Jammy to install prerequisite packages before building DPDK. Best regards Heinrich >> >>> >>> NOTE: This work was sponsored by StarFive and SiFive which is signified by >>> "Sponsored-by:" sign-offs in each commit message. After discussing it >>> with Thomas Monjalon it seemed a better choice than "Suggested-by" which >>> does not fully convey the nature of involvement. However it makes >>> Linux checkpatch unhappy so I'm not sure if I shouldn't change the >>> sign-offs. >>> >>> NOTE2: I have added maintainers for each commit based on MAINTAINERS file. >>> However some modules (l3fwd, net/tap and cpuflags unit tests) do not have >>> any maintainers assigned, hence I've targeted dev@dpdk.org mailing list as >>> if it was a commit adding new files. >>> >>> Michal Mazurek (3): >>> lpm: add a scalar version of lookupx4 function >>> eal: add initial support for RISC-V architecture >>> test/cpuflags: add test for RISC-V cpu flag >>> >>> Stanislaw Kardach (8): >>> examples/l3fwd: fix scalar LPM compilation >>> net/ixgbe: enable vector stubs for RISC-V >>> net/memif: set memfd syscall ID on RISC-V >>> net/tap: set BPF syscall ID for RISC-V >>> examples/l3fwd: enable RISC-V operation >>> test/ring: disable problematic tests for RISC-V >>> devtools: add RISC-V to test-meson-builds.sh >>> test/hash: report non HTM numbers for single r/w >>> >>> MAINTAINERS | 6 + >>> app/test/test_cpuflags.c | 81 ++++++++++ >>> app/test/test_hash_readwrite.c | 8 +- >>> app/test/test_ring.c | 8 + >>> app/test/test_xmmt_ops.h | 16 ++ >>> config/meson.build | 2 + >>> config/riscv/meson.build | 148 ++++++++++++++++++ >>> config/riscv/riscv64_linux_gcc | 17 ++ >>> config/riscv/riscv64_sifive_u740_linux_gcc | 19 +++ >>> devtools/test-meson-builds.sh | 6 + >>> doc/guides/contributing/design.rst | 2 +- >>> .../linux_gsg/cross_build_dpdk_for_riscv.rst | 125 +++++++++++++++ >>> doc/guides/linux_gsg/index.rst | 1 + >>> doc/guides/nics/features.rst | 5 + >>> doc/guides/nics/features/default.ini | 1 + >>> doc/guides/nics/features/ixgbe.ini | 1 + >>> doc/guides/rel_notes/release_22_07.rst | 29 ++++ >>> drivers/net/i40e/meson.build | 6 + >>> drivers/net/ixgbe/ixgbe_rxtx.c | 4 +- >>> drivers/net/memif/rte_eth_memif.h | 2 + >>> drivers/net/tap/tap_bpf.h | 2 + >>> examples/l3fwd/l3fwd_em.c | 8 + >>> examples/l3fwd/l3fwd_fib.c | 2 + >>> examples/l3fwd/l3fwd_lpm.c | 2 +- >>> lib/eal/riscv/include/meson.build | 23 +++ >>> lib/eal/riscv/include/rte_atomic.h | 52 ++++++ >>> lib/eal/riscv/include/rte_byteorder.h | 44 ++++++ >>> lib/eal/riscv/include/rte_cpuflags.h | 55 +++++++ >>> lib/eal/riscv/include/rte_cycles.h | 103 ++++++++++++ >>> lib/eal/riscv/include/rte_io.h | 21 +++ >>> lib/eal/riscv/include/rte_mcslock.h | 18 +++ >>> lib/eal/riscv/include/rte_memcpy.h | 63 ++++++++ >>> lib/eal/riscv/include/rte_pause.h | 31 ++++ >>> lib/eal/riscv/include/rte_pflock.h | 17 ++ >>> lib/eal/riscv/include/rte_power_intrinsics.h | 22 +++ >>> lib/eal/riscv/include/rte_prefetch.h | 50 ++++++ >>> lib/eal/riscv/include/rte_rwlock.h | 44 ++++++ >>> lib/eal/riscv/include/rte_spinlock.h | 67 ++++++++ >>> lib/eal/riscv/include/rte_ticketlock.h | 21 +++ >>> lib/eal/riscv/include/rte_vect.h | 55 +++++++ >>> lib/eal/riscv/meson.build | 11 ++ >>> lib/eal/riscv/rte_cpuflags.c | 122 +++++++++++++++ >>> lib/eal/riscv/rte_cycles.c | 77 +++++++++ >>> lib/eal/riscv/rte_hypervisor.c | 13 ++ >>> lib/eal/riscv/rte_power_intrinsics.c | 56 +++++++ >>> lib/lpm/meson.build | 1 + >>> lib/lpm/rte_lpm.h | 4 +- >>> lib/lpm/rte_lpm_scalar.h | 122 +++++++++++++++ >>> meson.build | 2 + >>> 49 files changed, 1588 insertions(+), 7 deletions(-) >>> create mode 100644 config/riscv/meson.build >>> create mode 100644 config/riscv/riscv64_linux_gcc >>> create mode 100644 config/riscv/riscv64_sifive_u740_linux_gcc >>> create mode 100644 doc/guides/linux_gsg/cross_build_dpdk_for_riscv.rst >>> create mode 100644 lib/eal/riscv/include/meson.build >>> create mode 100644 lib/eal/riscv/include/rte_atomic.h >>> create mode 100644 lib/eal/riscv/include/rte_byteorder.h >>> create mode 100644 lib/eal/riscv/include/rte_cpuflags.h >>> create mode 100644 lib/eal/riscv/include/rte_cycles.h >>> create mode 100644 lib/eal/riscv/include/rte_io.h >>> create mode 100644 lib/eal/riscv/include/rte_mcslock.h >>> create mode 100644 lib/eal/riscv/include/rte_memcpy.h >>> create mode 100644 lib/eal/riscv/include/rte_pause.h >>> create mode 100644 lib/eal/riscv/include/rte_pflock.h >>> create mode 100644 lib/eal/riscv/include/rte_power_intrinsics.h >>> create mode 100644 lib/eal/riscv/include/rte_prefetch.h >>> create mode 100644 lib/eal/riscv/include/rte_rwlock.h >>> create mode 100644 lib/eal/riscv/include/rte_spinlock.h >>> create mode 100644 lib/eal/riscv/include/rte_ticketlock.h >>> create mode 100644 lib/eal/riscv/include/rte_vect.h >>> create mode 100644 lib/eal/riscv/meson.build >>> create mode 100644 lib/eal/riscv/rte_cpuflags.c >>> create mode 100644 lib/eal/riscv/rte_cycles.c >>> create mode 100644 lib/eal/riscv/rte_hypervisor.c >>> create mode 100644 lib/eal/riscv/rte_power_intrinsics.c >>> create mode 100644 lib/lpm/rte_lpm_scalar.h >>> >>