From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yw1-f50.google.com (mail-yw1-f50.google.com [209.85.161.50]) by dpdk.org (Postfix) with ESMTP id 92D365B3A for ; Fri, 1 Mar 2019 03:23:13 +0100 (CET) Received: by mail-yw1-f50.google.com with SMTP id k14so12844233ywe.4 for ; Thu, 28 Feb 2019 18:23:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=l8B+ijPKflMByW0geITIAmwq/R7Ip0XQzTqbvMAT+U0=; b=KkN3KZIl9gOGHBzSG8uSpgRP9DVFN2ayxAwgFSwL/JV+chlQon/uAbLnsZ6waXg88g J/MhCFU804dwbu9S6MdGUZ+VbBdRYfqtADExzAXRdpsypC/SlOHUJ7DbZ+Pku2BX8Ydy 0D+cGeCwrTGx0OtBf66SNgGMxaT0VL/GBsPo+5YMoYI2mLVJ9q2GzxHvo2H5N/qe8lbZ 5teo3CeT6YpX0mdzwBopm9LakikPpkKLNbvQZ8cmdTQ9F52Gm7t1OdbFSAXrjWaY7hj9 gNlEQheOzZlx3Iipzumxykqz/JvEEm27xlRy++MvmUlPz/zfD03xKyNMWuUodEAS1/bG CXIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=l8B+ijPKflMByW0geITIAmwq/R7Ip0XQzTqbvMAT+U0=; b=UQluMA2CwCr5sM2x4E8nVJzcQ29EzN3u1RAVFv5X0oV6+1LskTV4KttSK2Ip9TpYZm Uf0MLMthzmsh1Lh1tA4ogAoo3HSOByXdvyEp57Ol5qJUOFnP3+pm6rgNtPFYA5FeWfqP utyaZ/44UVQM5wqdIb6BhhKVACOu5BB6oViKkN2JoN36spfY+qbUUIWYoTyOcn2nRc7E Bfa5HLTdutF3O8o4NNfSAWsP0O+JhEDMLfaKSVN23qpTF4Lq1tZ5eHO/vexDwTu+rcR6 VYzrWo+GBWuRYjMGmxILkMy5FSzdqgIHTxDOl8MSelgpV4KBJpbrh3Y1C6UuXd3Fqg2V ouKg== X-Gm-Message-State: APjAAAWc+Op7FneuNwVjszCZcisouW/i0IM0M4V1nRyK+w1u1fGGYTvk MnW8Pi+9pOKQ2HAD1S+jdG3n4YvWyy5zca1KKwA= X-Google-Smtp-Source: APXvYqxqZPXWXucZKwvrWJhBYUgM6bpq2I4rxnCJMMX3B+l26u060JiN6A52DJkMuste8l/gTmBG4j93mdkb041eBWs= X-Received: by 2002:a81:9bd0:: with SMTP id s199mr1742956ywg.446.1551406992660; Thu, 28 Feb 2019 18:23:12 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Cliff Burdick Date: Thu, 28 Feb 2019 18:23:01 -0800 Message-ID: To: Arvind Narayanan Cc: users Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] rte_flow / hw-offloading is degrading performance when testing @ 100G X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Mar 2019 02:23:14 -0000 What size packets are you using? I've only steered to 2 rx queues by IP dst match, and was able to hit 100Gbps. That's with a 4KB jumboframe. On Thu, Feb 28, 2019, 17:42 Arvind Narayanan wrote: > Hi, > > I am using DPDK 18.11 on Ubuntu 18.04, with Mellanox Connect X-5 100G > EN (MLNX_OFED_LINUX-4.5-1.0.1.0-ubuntu18.04-x86_64). > Packet generator: t-rex 2.49 running on another machine. > > I am able to achieve 100G line rate with l3fwd application (fr sz 64B) > using the parameters suggested in their performance report. > ( > https://fast.dpdk.org/doc/perf/DPDK_18_11_Mellanox_NIC_performance_report.pdf > ) > > However, as soon as I install rte_flow rules to steer packets to > different queues and/or use rte_flow's mark action, the throughput > reduces to ~41G. I also modified DPDK's flow_filtering example > application, and am getting the same reduced throughput of around 41G > out of 100G. But without rte_flow, it goes to 100G. > > I didn't change any OS/Kernel parameters to test l3fwd or the > application that uses rte_flow. I also ensure the application is > numa-aware and use 20 cores to handle 100G traffic. > > Upon further investigation (using Mellanox NIC counters), the drop in > throughput is due to mbuf allocation errors. > > Is such performance degradation normal when performing hw-acceleration > using rte_flow? > Has anyone tested throughput performance using rte_flow @ 100G? > > Its surprising to see hardware offloading is degrading the > performance, unless I am doing something wrong. > > Thanks, > Arvind >