From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6D8F9A0545; Thu, 26 Nov 2020 02:21:20 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E2B52C960; Thu, 26 Nov 2020 02:21:17 +0100 (CET) Received: from dish-sg.nttdocomo.co.jp (dish-sg.nttdocomo.co.jp [202.19.227.74]) by dpdk.org (Postfix) with ESMTP id 3DB9AC95E for ; Thu, 26 Nov 2020 02:21:15 +0100 (CET) X-dD-Source: Outbound Received: from zssg-mailmd102.ddreams.local (zssg-mailmd900.ddreams.local [10.160.172.63]) by zssg-mailou102.ddreams.local (Postfix) with ESMTP id 09A6C120146; Thu, 26 Nov 2020 10:21:13 +0900 (JST) Received: from t131sg-mailcc12.ddreams.local (t131sg-mailcc12.ddreams.local [100.66.31.87]) by zssg-mailmd102.ddreams.local (dDREAMS) with ESMTP id <0QKD002VVQFC8L70@dDREAMS>; Thu, 26 Nov 2020 10:21:12 +0900 (JST) Received: from t131sg-mailcc11 (localhost [127.0.0.1]) by t131sg-mailcc12.ddreams.local (unknown) with SMTP id 0AQ1LCWY041022; Thu, 26 Nov 2020 10:21:12 +0900 Received: from zssg-mailmf105.ddreams.local (unknown [127.0.0.1]) by zssg-mailmf105.ddreams.local (Postfix) with ESMTP id 28ECF7E603C; Thu, 26 Nov 2020 10:21:00 +0900 (JST) Received: from zssg-mailmf105.ddreams.local (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 25AA38E605D; Thu, 26 Nov 2020 10:21:00 +0900 (JST) Received: from localhost (unknown [127.0.0.1]) by IMSVA (Postfix) with SMTP id 24D9E8E6066; Thu, 26 Nov 2020 10:21:00 +0900 (JST) X-IMSS-HAND-OFF-DIRECTIVE: localhost:10026 Received: from zssg-mailmf105.ddreams.local (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E26028E6058; Thu, 26 Nov 2020 10:20:59 +0900 (JST) Received: from zssg-mailua106.ddreams.local (unknown [10.160.172.62]) by zssg-mailmf105.ddreams.local (Postfix) with ESMTP; Thu, 26 Nov 2020 10:20:59 +0900 (JST) Received: from [10.87.198.18] (unknown [10.160.183.129]) by zssg-mailua106.ddreams.local (dDREAMS) with ESMTPA id <0QKD00Y6SQEQB9H0@dDREAMS>; Thu, 26 Nov 2020 10:20:50 +0900 (JST) Date: Thu, 26 Nov 2020 10:20:49 +0900 From: Hideyuki Yamashita In-reply-to: <98CBD80474FA8B44BF855DF32C47DC35C6144E@smartserver.smartshare.dk> References: <20201125143930.C6ED.17218CA3@ntt-tx.co.jp_1> <98CBD80474FA8B44BF855DF32C47DC35C6144E@smartserver.smartshare.dk> Message-id: <20201126102049.C6F8.17218CA3@ntt-tx.co.jp_1> MIME-version: 1.0 Content-type: text/plain; charset=ISO-2022-JP Content-transfer-encoding: 7bit X-Mailer: Becky! ver. 2.75.01 [ja] X-TM-AS-GCONF: 00 To: Morten Brorup Cc: dev@dpdk.org Subject: Re: [dpdk-dev] NTT TechnoCross roadmap for 21.02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hello Morten, Thanks for your giving me your valuable feedback. Please see inline tagged with [Hideyuki]. > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Hideyuki Yamashita > > Sent: Wednesday, November 25, 2020 6:40 AM > > > > Hello, > > > > Following are the work items planned for 21.02 from NTT TechnoCross: > > I will try to post patch set after 20.11 is released. > > > > --- > > 1) Introduce API stats function > > In general, DPDK application consumes CPU usage because it polls > > incoming packets using rx_burst API in infinite loop. > > This makes difficult to estimate how much CPU usage is really > > used to send/receive packets by the DPDK application. > > > > For example, even if no incoming packets arriving, CPU usage > > looks nearly 100% when observed by top command. > > > > It is beneficial if developers can observe real CPU usage of the > > DPDK application. > > Such information can be exported to monitoring application like > > prometheus/graphana and shows CPU usage graphically. > > This would be very beneficial. > > Unfortunately, this seems to be not so simple for applications like the SmartShare StraightShaper, which is not a simple packet forwarding application, but has multiple pipeline stages. Our application also keeps some packets in queues for shaping purposes, so the number of packets transmitted does not match the number of packets received within some time interval. [Hideyuki] Thanks. I share the same view with you. DPDK application varies and not all applications "simply forward incoming packets". So I think maybe target applications are limited. Though I believe this enhancement is useful for those applications still. > > > > To achieve above, this patch set provides apistats functionality. > > apistats provides the followiing two counters for each lcore. > > - rx_burst_counts[RTE_MAX_LCORE] > > - tx_burst_counts[RTE_MAX_LCORE] > > Those accumulates rx_burst/tx_burst counts since the application > > starts. > > > > By using those values, developers can roughly estimate CPU usage. > > Let us assume a DPDK application is simply forwarding packets. > > It calls tx_burst only if it receive packets. > > If rx_burst_counts=1000 and tx_burst_count=1000 during certain > > period of time, one can assume CPU usage is 100%. > > If rx_burst_counts=1000 and tx_burst_count=100 during certain > > period of time, one can assume CPU usage is 10%. > > Here we assumes that tx_burst_count equals counts which rx_burst > > function > > really receives incoming packets. > > I am not sure I understand what is being counted in these counters. The number of packets in the bursts, or the number of invocations of the rx_burst/tx_burst functions. [Hideyuki] Latter. I think exsisting mechanism may store number of packets. (maybe I am wrong) > > Here are some data from our purpose built profiler, illustrating how nonlinear this really is. These data are from a SmartShare appliance in live production at an ISP. I hope you find it useful: > > Rx_burst uses ca. 40 CPU cycles if there are no packets, ca. 260 cycles if there is one packet, and down to ca. 40 cycles per packet for a burst of many packets. > > Tx_burst uses ca. 350 cycles for one packet, and down to ca. 20 cycles per packet for a burst of many packets. [Hideyuki] Thanks for your sharing useful info! Ah, I realized that consumption of CPU cycle is not linear like following. 0 packet receive -> 0 cycle 1 packet receive -> 1 cycle 10 packets receive -> 10 cycle It is very interesting. Thanks for your information. I will keep your information in my mind. > One of our intermediate pipeline stages (which not is not receiving or transmitting packets, only processing them) uses ca. 150 cycles for a burst of one packet, and down to ca. 110 cycles for a burst of many packets. > > Nevertheless, your suggested API might be usable by simple ingress->routing->egress applications. So don’t let me discourage you! [Hideyuki] Thanks for your supporting my idea. Yes, I agree with you that for simple forwarding applications, this enhancement might be useful to monitor the cpu usage "roughly". BR, Hideyuki Yamashita NTT TechnoCross >