From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9C3DB42F6C for ; Fri, 28 Jul 2023 16:08:08 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0DE8440685; Fri, 28 Jul 2023 16:08:08 +0200 (CEST) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) by mails.dpdk.org (Postfix) with ESMTP id 468BC4021E for ; Fri, 28 Jul 2023 16:08:07 +0200 (CEST) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id E3C2C5C018E; Fri, 28 Jul 2023 10:08:06 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Fri, 28 Jul 2023 10:08:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm3; t= 1690553286; x=1690639686; bh=0d0pTjqUvSqluN1XT0l/coNpIlwwCC3ITFW 8gBekNFA=; b=e33mgoF8kD+dPq+RRJpB+i9mskUnoD7WKGSc4hg9zFHHuLQrz4u mTyqHW7wTTYzT2FgevenpWVxrdhSmYNFkWQj9FCtNUZJDIBRu7HBVSbKuaxKSTtO lqwPZohD6T+PvRWvzFsg74bSlmwjF/XPwfxE+n0phtTpCZ6ahnOR99XDpAFARvor vSfFhwevxOA4b1yI7l6aA0aASSCy2CSPKv6VnLtht/J0d/dT4AYhtoGkchG/6g+6 ff+ZlQK7s+QSEcByjfwGEFub9s0Tj2BoDK7QNYyFctdkEWYq5PcOhJaoAnq+IzxA MmctpRR0hxJhuSaM8McOr6ZLmAOaJPrkbqw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1690553286; x=1690639686; bh=0d0pTjqUvSqluN1XT0l/coNpIlwwCC3ITFW 8gBekNFA=; b=fhIPCEBYnEaYl6oIS0rJRW3nw9PTcildDW9oY76gA/uXSYFWGLP +IYQMpIZ8Xc2f0moaz6rpTdspA/nPS/jQrrdlB9Tzh7Wo9Gv+yTl4ObgDD2HUYyx hkTfsUABxFT0xhGgzO+z504gtKdkD/ji83uif2tfk+A4d6TeUlpb49S2TI1hwtBa DB/sh41qhDRyDFxJNxMKMgEssTNtDb/H1qKb4PE2lIN9qSl3XeV9+TlOAETqpt81 iGDJs7tlIV9B25ZqvGVX70gd7NoKRED389qdpfqWCSDHcQS5oaSOi5354MNutdQZ VBp4iq4s4NL4pg0ptW5gmsouiB4X/JV49LA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrieeigdejtdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvfevufffkfgjfhgggfgtsehtqhertddttdejnecuhfhrohhmpefvhhhomhgr shcuofhonhhjrghlohhnuceothhhohhmrghssehmohhnjhgrlhhonhdrnhgvtheqnecugg ftrfgrthhtvghrnhepheehteffveevvdejfffgkedtueejtdeivedttedthedvueektdfh hfejtdehhfdvnecuffhomhgrihhnpehrvggrughthhgvughotghsrdhiohdprhigmhhoug gvrdhmqhenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhm pehthhhomhgrshesmhhonhhjrghlohhnrdhnvght X-ME-Proxy: Feedback-ID: i47234305:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 28 Jul 2023 10:08:05 -0400 (EDT) From: Thomas Monjalon To: shiv chittora Cc: Bing Zhao , users@dpdk.org, maxime.coquelin@redhat.com Subject: Re: Enable RSS for virtio application ( dpdk version 21.11) Date: Fri, 28 Jul 2023 16:08:04 +0200 Message-ID: <1831207.atdPhlSkOF@thomas> In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org You may need vhost running in the Linux kernel with some BPF code. There is a documentation about eBPF RSS: https://qemu.readthedocs.io/en/latest/devel/ebpf_rss.html 26/07/2023 09:32, shiv chittora: > Thanks Bing for quick response. >=20 > The virtio driver version 1.0.0 is included in the Linux kernel version 4= =2E9 > that powers VM. >=20 > ethtool -i eth1 > driver: virtio_net > version: 1.0.0 > firmware-version: > expansion-rom-version: > bus-info: 0000:00:04.0 >=20 > Nutanix document stats that "Ensure the AHV UVM is running the latest > Nutanix VirtIO driver package. Nutanix VirtIO 1.1.6 or higher is required > for RSS support. " Linux kernel version: 5.4 and later will have Virtio > 1.1.6. >=20 > Since the programme is built on the dpdk, the PMD driver will use the eth > interface rather than the one that the kernel provides. I apologise if I'm > mistaken. RSS is supported by the dpdk PMD version in use. >=20 > Because of the client-centric nature of this application, upgrading the > kernel will be challenging. >=20 > Do you believe that the only option is to upgrade the vm kernel version? >=20 > Thanks , > Shiv >=20 > On Wed, Jul 26, 2023 at 12:33=E2=80=AFPM Bing Zhao wro= te: >=20 > > IIRC, the =E2=80=9CVIRTIO_NET_F_RSS=E2=80=9D is some capability reporte= d and decided > > during the driver setup/communication stage. It is mostly like that your > > libs/drivers running on the host for the VM does not support this featu= re. > > > > Have you tried to update the versions of VM or the package/lib of VirtIO > > for this VM? > > > > > > > > *From:* shiv chittora > > *Sent:* Wednesday, July 26, 2023 1:05 PM > > *To:* users@dpdk.org > > *Subject:* Enable RSS for virtio application ( dpdk version 21.11) > > > > > > > > *External email: Use caution opening links or attachments* > > > > > > > > I'm using a Nutanix virtual machine to run a DPDK(Version 21.11)-based > > application. > > Application is failing during rte_eth_dev_configure . For our applicati= on, > > RSS support is required. > > > > eth_config.rxmode.mq_mode =3D ETH_MQ_RX_RSS; > > static uint8_t hashKey[] =3D { > > 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, > > 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, > > 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, > > 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, > > 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, > > }; > > > > eth_config.rx_adv_conf.rss_conf.rss_key =3D hashKey; > > eth_config.rx_adv_conf.rss_conf.rss_key_len =3D sizeof(hashKey); > > eth_config.rx_adv_conf.rss_conf.rss_hf =3D 260 > > > > > > > > With the aforementioned RSS configuration, the application is not coming > > up. The same application runs without any issues on a VMware virtual > > machine. > > > > When I set > > > > eth_config.rxmode.mq_mode =3D ETH_MQ_RX_NONE > > eth_config.rx_adv_conf.rss_conf.rss_hf =3D 0 > > > > Application starts working fine. Since we need RSS support for our > > application I cannot set eth_config.rxmode.mq_mode =3D ETH_MQ_RX_NONE. > > > > I looked at the DPDK 21.11 release notes, and it mentions that virtio_n= et > > supports RSS support. > > > > > > In this application traffic is tapped to capture port. I have also crea= ted > > two queues using ACLI comments. > > > > vm.nic_create nutms1-ms type=3DkNetworkFunctionNic > > network_function_nic_type=3DkTap queues=3D2 > > > > vm.nic_get testvm > > xx:xx:xx:xx:xx:xx { > > mac_addr: "xx:xx:xx:xx:xx:xx" > > network_function_nic_type: "kTap" > > network_type: "kNativeNetwork" > > queues: 2 > > type: "kNetworkFunctionNic" > > uuid: "9c26c704-bcb3-4483-bdaf-4b64bb9233ef" > > } > > > > > > Additionally, I've turned on dpdk logging. PFB the dpdk log's output. > > > > EAL: PCI device 0000:00:05.0 on NUMA socket 0 > > EAL: probe driver: 1af4:1000 net_virtio > > EAL: Probe PCI driver: net_virtio (1af4:1000) device: 0000:00:05.0 (soc= ket > > 0) > > EAL: PCI memory mapped at 0x940000000 > > EAL: PCI memory mapped at 0x940001000 > > virtio_read_caps(): [98] skipping non VNDR cap id: 11 > > virtio_read_caps(): [84] cfg type: 5, bar: 0, offset: 0000, len: 0 > > virtio_read_caps(): [70] cfg type: 2, bar: 4, offset: 3000, len: 4096 > > virtio_read_caps(): [60] cfg type: 4, bar: 4, offset: 2000, len: 4096 > > virtio_read_caps(): [50] cfg type: 3, bar: 4, offset: 1000, len: 4096 > > virtio_read_caps(): [40] cfg type: 1, bar: 4, offset: 0000, len: 4096 > > virtio_read_caps(): found modern virtio pci device. > > virtio_read_caps(): common cfg mapped at: 0x940001000 > > virtio_read_caps(): device cfg mapped at: 0x940003000 > > virtio_read_caps(): isr cfg mapped at: 0x940002000 > > virtio_read_caps(): notify base: 0x940004000, notify off multiplier: 4 > > vtpci_init(): modern virtio pci detected. > > virtio_ethdev_negotiate_features(): guest_features before negotiate =3D > > 8000005f10ef8028 > > virtio_ethdev_negotiate_features(): host_features before negotiate =3D > > 130ffffa7 > > virtio_ethdev_negotiate_features(): features after negotiate =3D 110ef8= 020 > > virtio_init_device(): PORT MAC: 50:6B:8D:A9:09:62 > > virtio_init_device(): link speed =3D -1, duplex =3D 1 > > virtio_init_device(): config->max_virtqueue_pairs=3D2 > > virtio_init_device(): config->status=3D1 > > virtio_init_device(): PORT MAC: 50:6B:8D:A9:09:62 > > virtio_init_queue(): setting up queue: 0 on NUMA node 0 > > virtio_init_queue(): vq_size: 256 > > virtio_init_queue(): vring_size: 10244, rounded_vring_size: 12288 > > virtio_init_queue(): vq->vq_ring_mem: 0x7fffab000 > > virtio_init_queue(): vq->vq_ring_virt_mem: 0x17ffab000 > > virtio_init_vring(): >> > > modern_setup_queue(): queue 0 addresses: > > modern_setup_queue(): desc_addr: 7fffab000 > > modern_setup_queue(): aval_addr: 7fffac000 > > modern_setup_queue(): used_addr: 7fffad000 > > modern_setup_queue(): notify addr: 0x940004000 (notify offset: 0) > > virtio_init_queue(): setting up queue: 1 on NUMA node 0 > > virtio_init_queue(): vq_size: 256 > > virtio_init_queue(): vring_size: 10244, rounded_vring_size: 12288 > > virtio_init_queue(): vq->vq_ring_mem: 0x7fffa6000 > > virtio_init_queue(): vq->vq_ring_virt_mem: 0x17ffa6000 > > virtio_init_vring(): >> > > modern_setup_queue(): queue 1 addresses: > > modern_setup_queue(): desc_addr: 7fffa6000 > > modern_setup_queue(): aval_addr: 7fffa7000 > > modern_setup_queue(): used_addr: 7fffa8000 > > modern_setup_queue(): notify addr: 0x940004004 (notify offset: 1) > > virtio_init_queue(): setting up queue: 2 on NUMA node 0 > > virtio_init_queue(): vq_size: 256 > > virtio_init_queue(): vring_size: 10244, rounded_vring_size: 12288 > > virtio_init_queue(): vq->vq_ring_mem: 0x7fff98000 > > virtio_init_queue(): vq->vq_ring_virt_mem: 0x17ff98000 > > virtio_init_vring(): >> > > modern_setup_queue(): queue 2 addresses: > > modern_setup_queue(): desc_addr: 7fff98000 > > modern_setup_queue(): aval_addr: 7fff99000 > > modern_setup_queue(): used_addr: 7fff9a000 > > modern_setup_queue(): notify addr: 0x940004008 (notify offset: 2) > > virtio_init_queue(): setting up queue: 3 on NUMA node 0 > > virtio_init_queue(): vq_size: 256 > > virtio_init_queue(): vring_size: 10244, rounded_vring_size: 12288 > > virtio_init_queue(): vq->vq_ring_mem: 0x7fff93000 > > virtio_init_queue(): vq->vq_ring_virt_mem: 0x17ff93000 > > virtio_init_vring(): >> > > modern_setup_queue(): queue 3 addresses: > > modern_setup_queue(): desc_addr: 7fff93000 > > modern_setup_queue(): aval_addr: 7fff94000 > > modern_setup_queue(): used_addr: 7fff95000 > > modern_setup_queue(): notify addr: 0x94000400c (notify offset: 3) > > virtio_init_queue(): setting up queue: 4 on NUMA node 0 > > virtio_init_queue(): vq_size: 64 > > virtio_init_queue(): vring_size: 4612, rounded_vring_size: 8192 > > virtio_init_queue(): vq->vq_ring_mem: 0x7fff87000 > > virtio_init_queue(): vq->vq_ring_virt_mem: 0x17ff87000 > > virtio_init_vring(): >> > > modern_setup_queue(): queue 4 addresses: > > modern_setup_queue(): desc_addr: 7fff87000 > > modern_setup_queue(): aval_addr: 7fff87400 > > modern_setup_queue(): used_addr: 7fff88000 > > modern_setup_queue(): notify addr: 0x940004010 (notify offset: 4) > > eth_virtio_pci_init(): port 0 vendorID=3D0x1af4 deviceID=3D0x1000 > > EAL: Module /sys/module/vfio not found! error 2 (No such file or direct= ory) > > EAL: lib.telemetry log level changed from disabled to debug > > TELEMETRY: Attempting socket bind to path > > '/var/run/dpdk/rte/dpdk_telemetry.v2' > > TELEMETRY: Initial bind to socket '/var/run/dpdk/rte/dpdk_telemetry.v2' > > failed. > > TELEMETRY: Attempting unlink and retrying bind > > TELEMETRY: Socket creation and binding ok > > TELEMETRY: Telemetry initialized ok > > TELEMETRY: No legacy callbacks, legacy socket not created > > [Wed Jul 26 04:44:42 2023][ms_dpi: 28098] DPDK Initialised > > [Wed Jul 26 04:44:42 2023][ms_dpi: 28098] Finished DPDK logging session > > > > > > The following result is produced when testpmd runs the RSS configuration > > command. > > > > testpmd> port config all rss all > > Port 0 modified RSS hash function based on hardware > > support,requested:0x17f83fffc configured:0 > > Multi-queue RSS mode isn't enabled. > > Configuration of RSS hash at ethernet port 0 failed with error (95): > > Operation not supported. > > > > > > Any suggestions on how to enable RSS support in this situation would be > > greatly appreciated. > > > > Thank you for your assistance. > > >=20