From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ua0-f176.google.com (mail-ua0-f176.google.com [209.85.217.176]) by dpdk.org (Postfix) with ESMTP id 4A72C2C01 for ; Fri, 10 Mar 2017 14:16:19 +0100 (CET) Received: by mail-ua0-f176.google.com with SMTP id f54so107978114uaa.1 for ; Fri, 10 Mar 2017 05:16:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=infinite-io.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=GCgV2GuGDm3M5qSf2LWpwyVP53TmZ+wF4flRXB//5vw=; b=nogk+ma35tHHxIO5yIO2zw5CbOFycn8vKQ/KTbTIHrFWa4HmjbQC3rowf8J/hi79Rl InjwdK/guCIqoi7sMX4ysWXEqhZtIPmV71vq5Q3un07zRDufgC204ShS3fqkk3tGq1lZ Tt9K1KOKZeHE4DjVeiyWtCp/PKv67vyaEa2iGX/BIva6uL1e/rVC8xjFIAddhSVcy90u VRqXh12yrLaquuzM2IYoykOipln0j6sWzUjoRILUrd5y/HextV1Jf+uwvJ87/v3MvTCd 3V59i37+34lzE2a2MLB/UVjEA+6szphzlsbfTbrC4bfJB7iaXgWHc1motQH1fEnTIEuQ pZaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=GCgV2GuGDm3M5qSf2LWpwyVP53TmZ+wF4flRXB//5vw=; b=Y84ctZY8C7E7anTkxgpU0pZBCTQwAcyFsbBeJKiDRQ7FIi55VpK1fVu4TnYGgx9TrY vdHY038yToYoekhX5CK208/PNh0z5ZSvZuWHHvY5cnIOSSsTtEp2orVlLyWzs4y8460W 6QP1Sp7dmkxcqDYdEZCrcOpUyAh9BTZtojMOnM7LCh59nL4lsiiZnSrLRe9KRnVFERVH fgOfx3kwKpXcpV/bPsjGykGqs9tHj95J3uFX/AFCKu5UL7/uONL6cBjprDPdxRxhVjKk KZRmi63xhn+Zrz8mZX4G9e2DFjjbYD2Zapyov+fsBk9+muRAwKe/LySUAY/vyuktEfqo isZA== X-Gm-Message-State: AMke39kSLodXqMrKrMDrwQWYDUMs+sDJivoMdL+GtF7P46HrmoyFp+KJEYYquB6F+Wc6AkjaKXdahPTkPO+vPA== X-Received: by 10.31.110.138 with SMTP id j132mr7488343vkc.103.1489151778551; Fri, 10 Mar 2017 05:16:18 -0800 (PST) MIME-Version: 1.0 Received: by 10.103.87.214 with HTTP; Fri, 10 Mar 2017 05:16:18 -0800 (PST) In-Reply-To: <3217044.DUek5hOmOi@xps13> References: <1483053795-8489-1-git-send-email-s.vyazmitinov@brain4net.com> <3217044.DUek5hOmOi@xps13> From: Jay Rolette Date: Fri, 10 Mar 2017 07:16:18 -0600 Message-ID: To: Thomas Monjalon Cc: Sergey Vyazmitinov , Ferruh Yigit , DPDK Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [PATCH] kni: fast data availability check in thread_single loop X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Mar 2017 13:16:19 -0000 On Fri, Mar 10, 2017 at 6:59 AM, Thomas Monjalon wrote: > 2017-01-18 07:11, Jay Rolette: > > On Wed, Jan 18, 2017 at 5:05 AM, Sergey Vyazmitinov < > > s.vyazmitinov@brain4net.com> wrote: > > > > > On Thu, Jan 12, 2017 at 12:29 AM, Ferruh Yigit > > > > wrote: > > > > > > > On 12/29/2016 11:23 PM, Sergey Vyazmitinov wrote: > > > > > This allow to significant reduces packets processing latency. > > > > > > > > > > Signed-off-by: Sergey Vyazmitinov > [...] > > > > > --- a/lib/librte_eal/linuxapp/kni/kni_misc.c > > > > > +++ b/lib/librte_eal/linuxapp/kni/kni_misc.c > > > > > @@ -45,6 +45,7 @@ MODULE_AUTHOR("Intel Corporation"); > > > > > MODULE_DESCRIPTION("Kernel Module for managing kni devices"); > > > > > > > > > > #define KNI_RX_LOOP_NUM 1000 > > > > > +#define KNI_RX_DATA_LOOP_NUM 2500 > > > > > > > > > > #define KNI_MAX_DEVICES 32 > > > > > > > > > > @@ -129,25 +130,39 @@ static struct pernet_operations kni_net_ops > = { > > > > > #endif > > > > > }; > > > > > > > > > > -static int > > > > > -kni_thread_single(void *data) > > > > > +static inline void > > > > > +kni_thread_single_rx_data_loop(struct kni_net *knet) > > > > > { > > > > > - struct kni_net *knet = data; > > > > > - int j; > > > > > struct kni_dev *dev; > > > > > + int i; > > > > > > > > > > - while (!kthread_should_stop()) { > > > > > - down_read(&knet->kni_list_lock); > > > > > - for (j = 0; j < KNI_RX_LOOP_NUM; j++) { > > > > > - list_for_each_entry(dev, > &knet->kni_list_head, > > > > list) { > > > > > + for (i = 0; i < KNI_RX_DATA_LOOP_NUM; ++i) { > > > > > > > > When there are multiple KNI interfaces, and lets assume there is > traffic > > > > too, this will behave like: > > > > > > > > KNI1x2500 data_packets + KNI2x2500 data_packets .... KNI10x2500 > > > > > > > > After data packets, KNI1 resp_packet + KNI2 resp_packets ... > > > > > > > > Won't this scenario also may cause latency? And perhaps jitter > according > > > > KNI interface traffic loads? > > > > > > > > This may be good for some use cases, but not sure if this is good for > > > all. > > > > > > > We can decrease KNI_RX_DATA_LOOP_NUM to some reasonable value. > > > I can make test to find lower bound. > > > Also, the point is in fast check for a new data in interface rx queue. > > > May be will be better add some kind of break after several kni_net_rx > > > calls. > > > Without them loop ends very quickly. > > > Anyway, this patch decrease average latency in my case from 4.5ms to > > > 0.011ms in ping test with 100000 packets. > > > > > > > If you were seeing latency of 4.5ms, then it is more likely a different > > issue. > > > > At the end of the loop where KNI is reading packets from the queue, it > > calls *schedule_timeout_interruptible()* with (by default) a 5us > timeout. > > However, that call just guarantees that the thread will sleep for AT > LEAST > > 5us. > > > > For most x86 Linux distros, HZ = 250 in the kernel, which works out to > 4ms. > > I'm reasonably certain the latency you are seeing is because the KNI > thread > > is sleeping and not getting woken up like you might expect. > > > > When you increased the number of loops happening before the sleep, you > > increased how long KNI spends before it sleeps and it happened to be long > > enough in your particular test to change your average latency. If you ran > > your test for a few minutes and built a histogram of ping times, I bet > > you'll see ~4ms of latency pop up regularly. > > > > More details from when I dug into this behavior previously: > > http://dpdk.org/ml/archives/dev/2015-June/018858.html > > No answer in this discussion. > Should we close it in patchwork? > I don't believe we should merge the patch. Jay