From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt0-f174.google.com (mail-qt0-f174.google.com [209.85.216.174]) by dpdk.org (Postfix) with ESMTP id EDBC3F97F for ; Wed, 18 Jan 2017 12:05:15 +0100 (CET) Received: by mail-qt0-f174.google.com with SMTP id x49so8156081qtc.2 for ; Wed, 18 Jan 2017 03:05:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=brain4net.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=uUpVgHIQ1jGdw2fFYF2amPNu2QCja+k8PyOw8qpF6cQ=; b=TILve98dpO1EEa/ECBui1MNoyv6i3hlhagfoUdcMt15AbmTgEYWKTT3mQi5RVXUCWN PxGDPV/gVsPzm7pasvyY0iH6uTMZLAXrqhzV+Woy/IkEdr45yOa5i5UhSSGB1R27sdHM w9cpBiMPnNuoWAZZ2GZIbpRr8XFAN0736rYpA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=uUpVgHIQ1jGdw2fFYF2amPNu2QCja+k8PyOw8qpF6cQ=; b=gGqif5zXrOtnEcKdfGIEcBddHLPPqi03FOCRrKlWAoJL5uOwtMebT8IL6p+NIZYb6W 64Ujf3vqT5xMT/evJ0gQafbC9wXwiulRyKmdnfv5/nLK+0HjQSyW2x9uKM6RBa2CAEF3 y7Rq7Ap/40KiOS0+NZUN3OkIURINJKtpaIp3+2kyh11gCoMi74CQ12t+zkFqZHYFxy0n XYTW/7nI05BocpSFy7PJt2drMWwboQd7zEepjkdddKzxcBWNzgE+jQfVewvWSvk35njq Yb7nWS1Mgt3ok2UCrlKO7TA6o5FdDaytu0g3IVGcDDrpyQoUOMQw6W7guN6ZUkHX5HTE hxoQ== X-Gm-Message-State: AIkVDXLWfqJlzUM4Jab6kmSCadAKI7+RoYhcOO+BqGy8hDT//xa1N4ACuQcKkayVMirQc5IeLXCUZ+rVgqzvtkQO X-Received: by 10.237.33.209 with SMTP id m17mr2350441qtc.59.1484737515279; Wed, 18 Jan 2017 03:05:15 -0800 (PST) MIME-Version: 1.0 Received: by 10.12.136.242 with HTTP; Wed, 18 Jan 2017 03:05:15 -0800 (PST) In-Reply-To: <4316d3de-8159-fd34-8515-f82ba33b31e8@intel.com> References: <1483053795-8489-1-git-send-email-s.vyazmitinov@brain4net.com> <4316d3de-8159-fd34-8515-f82ba33b31e8@intel.com> From: Sergey Vyazmitinov Date: Wed, 18 Jan 2017 18:05:15 +0700 Message-ID: To: Ferruh Yigit Cc: dev@dpdk.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [PATCH] kni: fast data availability check in thread_single loop X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2017 11:05:16 -0000 On Thu, Jan 12, 2017 at 12:29 AM, Ferruh Yigit wrote: > On 12/29/2016 11:23 PM, Sergey Vyazmitinov wrote: > > This allow to significant reduces packets processing latency. > > > > Signed-off-by: Sergey Vyazmitinov > > --- > > .../linuxapp/eal/include/exec-env/rte_kni_common.h | 6 ++++ > > lib/librte_eal/linuxapp/kni/kni_misc.c | 33 > ++++++++++++++++------ > > 2 files changed, 30 insertions(+), 9 deletions(-) > > > > diff --git a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > > index 09713b0..8183a8e 100644 > > --- a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > > +++ b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > > @@ -109,6 +109,12 @@ struct rte_kni_fifo { > > void *volatile buffer[]; /**< The buffer contains mbuf > pointers */ > > }; > > > > +static inline int > > +kni_fifo_empty(struct rte_kni_fifo *fifo) > > +{ > > + return fifo->write == fifo->read; > > +} > > + > > /* > > * The kernel image of the rte_mbuf struct, with only the relevant > fields. > > * Padding is necessary to assure the offsets of these fields > > diff --git a/lib/librte_eal/linuxapp/kni/kni_misc.c > b/lib/librte_eal/linuxapp/kni/kni_misc.c > > index 497db9b..4bf9bfa 100644 > > --- a/lib/librte_eal/linuxapp/kni/kni_misc.c > > +++ b/lib/librte_eal/linuxapp/kni/kni_misc.c > > @@ -45,6 +45,7 @@ MODULE_AUTHOR("Intel Corporation"); > > MODULE_DESCRIPTION("Kernel Module for managing kni devices"); > > > > #define KNI_RX_LOOP_NUM 1000 > > +#define KNI_RX_DATA_LOOP_NUM 2500 > > > > #define KNI_MAX_DEVICES 32 > > > > @@ -129,25 +130,39 @@ static struct pernet_operations kni_net_ops = { > > #endif > > }; > > > > -static int > > -kni_thread_single(void *data) > > +static inline void > > +kni_thread_single_rx_data_loop(struct kni_net *knet) > > { > > - struct kni_net *knet = data; > > - int j; > > struct kni_dev *dev; > > + int i; > > > > - while (!kthread_should_stop()) { > > - down_read(&knet->kni_list_lock); > > - for (j = 0; j < KNI_RX_LOOP_NUM; j++) { > > - list_for_each_entry(dev, &knet->kni_list_head, > list) { > > + for (i = 0; i < KNI_RX_DATA_LOOP_NUM; ++i) { > > When there are multiple KNI interfaces, and lets assume there is traffic > too, this will behave like: > > KNI1x2500 data_packets + KNI2x2500 data_packets .... KNI10x2500 > > After data packets, KNI1 resp_packet + KNI2 resp_packets ... > > Won't this scenario also may cause latency? And perhaps jitter according > KNI interface traffic loads? > > This may be good for some use cases, but not sure if this is good for all. > We can decrease KNI_RX_DATA_LOOP_NUM to some reasonable value. I can make test to find lower bound. Also, the point is in fast check for a new data in interface rx queue. May be will be better add some kind of break after several kni_net_rx calls. Without them loop ends very quickly. Anyway, this patch decrease average latency in my case from 4.5ms to 0.011ms in ping test with 100000 packets. > > > + list_for_each_entry(dev, &knet->kni_list_head, list) { > > + /* Burst dequeue from rx_q */ > > + if (!kni_fifo_empty((struct rte_kni_fifo > *)dev->rx_q)) { > > Do we need this check, since first thing in kni_net_rx_normal() is > checking if there is item in the queue? > > You right. Without that check latency is even less. > #ifdef RTE_KNI_VHOST > > kni_chk_vhost_rx(dev); > > #else > > kni_net_rx(dev); > > #endif > > - kni_net_poll_resp(dev); > > } > > } > > + } > > + list_for_each_entry(dev, &knet->kni_list_head, list) { > > + kni_net_poll_resp(dev); > > + } > > +} > > + > > +static int > > +kni_thread_single(void *data) > > +{ > > + struct kni_net *knet = data; > > + int j; > > + > > + while (!kthread_should_stop()) { > > + down_read(&knet->kni_list_lock); > > + for (j = 0; j < KNI_RX_LOOP_NUM; j++) > > + kni_thread_single_rx_data_loop(knet); > > up_read(&knet->kni_list_lock); > > #ifdef RTE_KNI_PREEMPT_DEFAULT > > /* reschedule out for a while */ > > > >