From: Jay Rolette <rolette@infinite.io>
To: Thomas Monjalon <thomas.monjalon@6wind.com>
Cc: Sergey Vyazmitinov <s.vyazmitinov@brain4net.com>,
Ferruh Yigit <ferruh.yigit@intel.com>, DPDK <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH] kni: fast data availability check in thread_single loop
Date: Fri, 10 Mar 2017 07:16:18 -0600 [thread overview]
Message-ID: <CADNuJVoeGGRRU8juoz__0YwqRn36Ly9zye8MBBjgiHLNbG0Thg@mail.gmail.com> (raw)
In-Reply-To: <3217044.DUek5hOmOi@xps13>
On Fri, Mar 10, 2017 at 6:59 AM, Thomas Monjalon <thomas.monjalon@6wind.com>
wrote:
> 2017-01-18 07:11, Jay Rolette:
> > On Wed, Jan 18, 2017 at 5:05 AM, Sergey Vyazmitinov <
> > s.vyazmitinov@brain4net.com> wrote:
> >
> > > On Thu, Jan 12, 2017 at 12:29 AM, Ferruh Yigit <ferruh.yigit@intel.com
> >
> > > wrote:
> > >
> > > > On 12/29/2016 11:23 PM, Sergey Vyazmitinov wrote:
> > > > > This allow to significant reduces packets processing latency.
> > > > >
> > > > > Signed-off-by: Sergey Vyazmitinov <s.vyazmitinov@brain4net.com>
> [...]
> > > > > --- a/lib/librte_eal/linuxapp/kni/kni_misc.c
> > > > > +++ b/lib/librte_eal/linuxapp/kni/kni_misc.c
> > > > > @@ -45,6 +45,7 @@ MODULE_AUTHOR("Intel Corporation");
> > > > > MODULE_DESCRIPTION("Kernel Module for managing kni devices");
> > > > >
> > > > > #define KNI_RX_LOOP_NUM 1000
> > > > > +#define KNI_RX_DATA_LOOP_NUM 2500
> > > > >
> > > > > #define KNI_MAX_DEVICES 32
> > > > >
> > > > > @@ -129,25 +130,39 @@ static struct pernet_operations kni_net_ops
> = {
> > > > > #endif
> > > > > };
> > > > >
> > > > > -static int
> > > > > -kni_thread_single(void *data)
> > > > > +static inline void
> > > > > +kni_thread_single_rx_data_loop(struct kni_net *knet)
> > > > > {
> > > > > - struct kni_net *knet = data;
> > > > > - int j;
> > > > > struct kni_dev *dev;
> > > > > + int i;
> > > > >
> > > > > - while (!kthread_should_stop()) {
> > > > > - down_read(&knet->kni_list_lock);
> > > > > - for (j = 0; j < KNI_RX_LOOP_NUM; j++) {
> > > > > - list_for_each_entry(dev,
> &knet->kni_list_head,
> > > > list) {
> > > > > + for (i = 0; i < KNI_RX_DATA_LOOP_NUM; ++i) {
> > > >
> > > > When there are multiple KNI interfaces, and lets assume there is
> traffic
> > > > too, this will behave like:
> > > >
> > > > KNI1x2500 data_packets + KNI2x2500 data_packets .... KNI10x2500
> > > >
> > > > After data packets, KNI1 resp_packet + KNI2 resp_packets ...
> > > >
> > > > Won't this scenario also may cause latency? And perhaps jitter
> according
> > > > KNI interface traffic loads?
> > > >
> > > > This may be good for some use cases, but not sure if this is good for
> > > all.
> > > >
> > > We can decrease KNI_RX_DATA_LOOP_NUM to some reasonable value.
> > > I can make test to find lower bound.
> > > Also, the point is in fast check for a new data in interface rx queue.
> > > May be will be better add some kind of break after several kni_net_rx
> > > calls.
> > > Without them loop ends very quickly.
> > > Anyway, this patch decrease average latency in my case from 4.5ms to
> > > 0.011ms in ping test with 100000 packets.
> > >
> >
> > If you were seeing latency of 4.5ms, then it is more likely a different
> > issue.
> >
> > At the end of the loop where KNI is reading packets from the queue, it
> > calls *schedule_timeout_interruptible()* with (by default) a 5us
> timeout.
> > However, that call just guarantees that the thread will sleep for AT
> LEAST
> > 5us.
> >
> > For most x86 Linux distros, HZ = 250 in the kernel, which works out to
> 4ms.
> > I'm reasonably certain the latency you are seeing is because the KNI
> thread
> > is sleeping and not getting woken up like you might expect.
> >
> > When you increased the number of loops happening before the sleep, you
> > increased how long KNI spends before it sleeps and it happened to be long
> > enough in your particular test to change your average latency. If you ran
> > your test for a few minutes and built a histogram of ping times, I bet
> > you'll see ~4ms of latency pop up regularly.
> >
> > More details from when I dug into this behavior previously:
> > http://dpdk.org/ml/archives/dev/2015-June/018858.html
>
> No answer in this discussion.
> Should we close it in patchwork?
>
I don't believe we should merge the patch.
Jay
prev parent reply other threads:[~2017-03-10 13:16 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-12-29 23:23 Sergey Vyazmitinov
2017-01-11 17:29 ` Ferruh Yigit
2017-01-18 11:05 ` Sergey Vyazmitinov
2017-01-18 12:35 ` Ferruh Yigit
2017-01-18 13:11 ` Jay Rolette
2017-03-10 12:59 ` Thomas Monjalon
2017-03-10 13:16 ` Jay Rolette [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CADNuJVoeGGRRU8juoz__0YwqRn36Ly9zye8MBBjgiHLNbG0Thg@mail.gmail.com \
--to=rolette@infinite.io \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
--cc=s.vyazmitinov@brain4net.com \
--cc=thomas.monjalon@6wind.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).