From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4FCC442FD0 for ; Fri, 4 Aug 2023 10:31:56 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 431A240EF1; Fri, 4 Aug 2023 10:31:56 +0200 (CEST) Received: from juno.mpi-klsb.mpg.de (juno.mpi-klsb.mpg.de [139.19.86.40]) by mails.dpdk.org (Postfix) with ESMTP id 0359540DDB for ; Wed, 2 Aug 2023 01:09:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mpi-sws.org ; s=mail201904; h=Content-Transfer-Encoding:Content-Type:Message-ID: References:In-Reply-To:Subject:Cc:To:From:Date:MIME-Version:sender:reply-to: content-id:content-description:resent-date:resent-from:resent-sender: resent-to:resent-cc:resent-message-id:list-id:list-help:list-unsubscribe: list-subscribe:list-post:list-owner:list-archive; bh=FAtUVBkd4nq6xB404ytB7IbtxqfH1Ig+xpvJNpfuzFM=; b=GPESvLgNWBjCyQX8l+3/2RbhWM ws7aHyCUwKRJvi/fP4N96DT0wfZ1u7s1riBteiU7OCuJvylCV8XrJ/cDhknRBDexE8YVhpE+poFcf BqBWc6cI7iRwy/sNd5iNKKDFsZisOQmNgXYiYmLR/RqqmGit2r6wuvTk0gr9wQ50kGb8=; Received: from zak.mpi-klsb.mpg.de ([139.19.205.27]:41450) by juno.mpi-klsb.mpg.de (envelope-from ) with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) id 1qQyUS-0001Sc-Sx; Wed, 02 Aug 2023 01:09:27 +0200 Received: from autoconfig.mpi-sws.org ([139.19.91.10]:36914 helo=mail.mpi-sws.org) by zak.mpi-klsb.mpg.de (envelope-from ) with esmtpsa (TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.94.2) id 1qQyUR-006wh7-Mx; Wed, 02 Aug 2023 01:09:19 +0200 MIME-Version: 1.0 Date: Wed, 02 Aug 2023 01:09:19 +0200 From: Matheus Stolet To: Thomas Monjalon Cc: users@dpdk.org, maxime.coquelin@redhat.com Subject: Re: Supporting RSS with DPDK in a VM In-Reply-To: <2160676.Icojqenx9y@thomas> References: <4c9482cb3259f1e75e19a5724a67ea8d@mpi-sws.org> <2160676.Icojqenx9y@thomas> User-Agent: Roundcube Webmail/1.4.13 Message-ID: X-Sender: mstolet@mpi-sws.org Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-RSPAMD-Score: -0.1 (/) X-RSPAMD-Report: Action: no action Symbol: ARC_NA(0.00) Symbol: RCVD_VIA_SMTP_AUTH(0.00) Symbol: MID_RHS_MATCH_FROM(0.00) Symbol: R_SPF_NEUTRAL(0.00) Symbol: FROM_HAS_DN(0.00) Symbol: TO_DN_SOME(0.00) Symbol: RCPT_COUNT_THREE(0.00) Symbol: MIME_GOOD(-0.10) Symbol: TO_MATCH_ENVRCPT_ALL(0.00) Symbol: DMARC_NA(0.00) Symbol: NEURAL_HAM(-0.00) Symbol: FROM_EQ_ENVFROM(0.00) Symbol: R_DKIM_NA(0.00) Symbol: MIME_TRACE(0.00) Symbol: ASN(0.00) Symbol: RCVD_COUNT_TWO(0.00) Symbol: RCVD_TLS_ALL(0.00) Message-ID: bed3f709da10ae83bb382a94b6d086a1@mpi-sws.org X-Mailman-Approved-At: Fri, 04 Aug 2023 10:31:55 +0200 X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sorry it this is a duplicate, but I believe that last time when I replied I forgot to cc the mailing list. First of all, thank you for the helpful info, but I have one more question! How do I get the packets distributed to multiple queues without RSS? I turned off RSS by setting ETH_MQ_RX_NONE. I configured multiple queues to be used by QEMU and OvS. Inside my DPDK application I checked the nb_rx_queues value inside the rte_eth_dev_info struct and it says I have 10 rx_queues and max_rx_queues for the virtio device that was bound to the dpdk application, so at least this initial configuration seems to be working and DPDK is detecting multiple queues. The issue I am having is that I am only receiving packets in rx queue 0. From some other similar questions it seems like that is the default and specified behaviour, so how do I also get the other rx queues to be used? I found some vague stuff about using the rte_flow API. I can see how the queue action can be used to move packets to specific queues. These packets could even be matched based on IP address and TCP ports, but I don't see how to somewhat uniformly distribute them between queues without using a hash and modulo operator (I believe the rte_flow API doesn't support that). On 2023-07-28 16:12, Thomas Monjalon wrote: > Hello, > > You can have packets distributed to multiple queues without RSS. > > If you really wants to enable the RSS algorithms, > it seems not supported for now with vhost_user. > It can be enabled with vhost running in the kernel: > https://qemu.readthedocs.io/en/latest/devel/ebpf_rss.html > > > 26/07/2023 00:58, Matheus Stolet: >> Hello, >> >> I am trying to run a DPDK application with RSS enabled so that I can >> have multiple rx queues. This application is running inside a VM. This >> VM is hosted by QEMU using KVM acceleration and OvS with DPDK and >> vhost-user are used in the backend. So to clarify things there are two >> DPDK portions to this. The first is the DPDK portion used by OvS that >> bypasses the host operating system. This is working fine. The other is >> a >> DPDK application inside the virtual machine that will bypass the guest >> operating system. This is where I am having trouble with. >> >> When I set rte_eth_conf.rxmode.mq_mode = ETH_MQ_RX_RSS in my >> application >> I get the following errors: >> Warning: NIC does not support all requested RSS hash functions. >> virtio_dev_configure(): RSS support requested but not supported by >> the >> device >> Port0 dev_configure = -95 >> >> I setup my VM in QEMU to have mq=on and queues=10. I also set the >> number >> of rx_queues when creating the vhost port using ovs to 10. Before >> binding the interface to DPDK, I used ethtool to verify if the network >> interface was actually setup to have multiple queues. >> >> Running the 'ethtool -l enps02' command yields the following output: >> Pre-set maximums: >> RX: 0 >> TX: 0 >> Other: 0 >> Combined: 10 >> Current hardware settings: >> RX: 0 >> TX: 0 >> Other: 0 >> Combined: 10 >> >> From my understanding the combined values indicate that the interface >> was properly setup to have multiple queues, so why am I getting the >> unsupported RSS error? Are there other configuration steps that I have >> to take to get this to work? Is RSS with DPDK not supported at all >> inside a VM at the moment? Perhaps the "Port0 dev_configure() = -95" >> error means something else? Without the receive side scaling turned on >> my application is not able to achieve the desired throughput and won't >> scale when I assign more cores to the application. >> >> Versions: >> VM >> DPDK: 21.11.4 >> Kernel: 5.4.0-148-generic >> Distribution: Ubuntu 20.04 >> >> Host >> DPDK: 21.11.4 >> QEMU: 8.0.90 >> OvS: 3.0.5 >> Kernel: 5.15.111.1.amd64-smp >> Distribution: Debian 11 >>