From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lj1-f175.google.com (mail-lj1-f175.google.com [209.85.208.175]) by dpdk.org (Postfix) with ESMTP id 110C25B12 for ; Fri, 1 Mar 2019 02:42:49 +0100 (CET) Received: by mail-lj1-f175.google.com with SMTP id a17so18922565ljd.4 for ; Thu, 28 Feb 2019 17:42:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=LC2ZlHnrdJT2l6hMBXmRpp50J/TVryHhPaZ8p1Gonrw=; b=nJXiYzGx0B7XL51gcfw9XMc9gPwjTnhnN1p46hA9eMFAz//tr8hWkhSI+YTTTKijti Gk7PSEWGvLXvT5++O/+TOYXVFfFwIjY9gNhfGU9ND7qXhy4ZnK/1S8a9uN8a6gaFOqXr pu7cHB0BppWB5VtEjDV6LWM0TFGSVNaR9EXT1OOfYankPIeE9JIty2eLT9mLSvDIjQQM 6lrt3LINqbwQMyALwF6j61g2+h5+OQsLpxGHWHlfyALNUg9tN2RKzXlzj6J/C01o/69U NxRxjzolJWqhwvBw1A4WHnKuebmMJoBjyhIoctHJSA2Nixr6R4j3js/xO8qS8qBgYEjH vq5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=LC2ZlHnrdJT2l6hMBXmRpp50J/TVryHhPaZ8p1Gonrw=; b=eLqVjtghPXnW1ux2HcHRTkVkJTNTozUN5V5D+6B3UKLi10gpiVuE836wFFBM90gWxl PVLrqAeosPNGap4Vm9KPVkhsizLIiiyY8y+EysXMWcFQK1RzogGly+B3DeeljqCrtvyH /WU6QxWiSFE+b8WGFLehHG6dhHEL5XEDPeTiDrXZARnD6VZJDQSPOU+t6IaktDJ/FvzB 8YnCfFz+B2g+4GprDtXJ/0RbBEtDvrVbaXfodYLFEzKxoPhfkDxPi02hFfhj8d2Ss7KH AYLOg9KfP8y/yZCU8FNj7GMKaGRvIT3MI5d19eg/QP3WYJcYI1T+3bLuvLwAC6MNGDWC tdwA== X-Gm-Message-State: APjAAAVeCxaXWkFu+oO5VDqy4h7cMZONBlkO2SElwlzFpASOmNTDSfO3 SEfpHli/BfKaxfy1kWtsLv6vPMIH4WPSAgTs0CNMBDpi X-Google-Smtp-Source: APXvYqwqz7/nqtJ71mo9wTZ7Wq89myY+sBfQW5t+wHQYBhP8me+L7ITgd1cZN2b0+qTyLcHjODScIdKMlse48lqKOJE= X-Received: by 2002:a2e:4:: with SMTP id 4mr1051990lja.165.1551404568125; Thu, 28 Feb 2019 17:42:48 -0800 (PST) MIME-Version: 1.0 From: Arvind Narayanan Date: Thu, 28 Feb 2019 19:42:37 -0600 Message-ID: To: users@dpdk.org Content-Type: text/plain; charset="UTF-8" Subject: [dpdk-users] rte_flow / hw-offloading is degrading performance when testing @ 100G X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Mar 2019 01:42:49 -0000 Hi, I am using DPDK 18.11 on Ubuntu 18.04, with Mellanox Connect X-5 100G EN (MLNX_OFED_LINUX-4.5-1.0.1.0-ubuntu18.04-x86_64). Packet generator: t-rex 2.49 running on another machine. I am able to achieve 100G line rate with l3fwd application (fr sz 64B) using the parameters suggested in their performance report. (https://fast.dpdk.org/doc/perf/DPDK_18_11_Mellanox_NIC_performance_report.pdf) However, as soon as I install rte_flow rules to steer packets to different queues and/or use rte_flow's mark action, the throughput reduces to ~41G. I also modified DPDK's flow_filtering example application, and am getting the same reduced throughput of around 41G out of 100G. But without rte_flow, it goes to 100G. I didn't change any OS/Kernel parameters to test l3fwd or the application that uses rte_flow. I also ensure the application is numa-aware and use 20 cores to handle 100G traffic. Upon further investigation (using Mellanox NIC counters), the drop in throughput is due to mbuf allocation errors. Is such performance degradation normal when performing hw-acceleration using rte_flow? Has anyone tested throughput performance using rte_flow @ 100G? Its surprising to see hardware offloading is degrading the performance, unless I am doing something wrong. Thanks, Arvind