From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A077CA04C0; Fri, 25 Sep 2020 12:28:15 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BADC11E4E8; Fri, 25 Sep 2020 12:28:14 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id D3BED1E4C3 for ; Fri, 25 Sep 2020 12:28:12 +0200 (CEST) IronPort-SDR: rM9LoUEZCIGhO/ZGLAaCnW/ZPKe8fdtd9oxxdE+Y1sk1GzswNP64vPPgQkj5PIoYhvnlQbAH9M uDEywl1bvxDg== X-IronPort-AV: E=McAfee;i="6000,8403,9754"; a="162392776" X-IronPort-AV: E=Sophos;i="5.77,301,1596524400"; d="scan'208";a="162392776" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 03:28:11 -0700 IronPort-SDR: otiPwW/FEwRlcogs2TAyTHrXttrXVoVNXOklNUhrkkjEJTStwuaLjF/RAA6mH498IR6TikygwN TaB6i9Q4LDbQ== X-IronPort-AV: E=Sophos;i="5.77,301,1596524400"; d="scan'208";a="487411219" Received: from bricha3-mobl.ger.corp.intel.com ([10.252.51.38]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 25 Sep 2020 03:28:09 -0700 Date: Fri, 25 Sep 2020 11:28:05 +0100 From: Bruce Richardson To: "Ananyev, Konstantin" Cc: Honnappa Nagarahalli , "Nicolau, Radu" , "Van Haaren, Harry" , "dev@dpdk.org" , "jerinj@marvell.com" , nd Message-ID: <20200925102805.GD923@bricha3-MOBL.ger.corp.intel.com> References: <20200908105211.10066-1-radu.nicolau@intel.com> <46118f3466274596a663d7d44abb680a@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Subject: Re: [dpdk-dev] [PATCH v1] event/sw: performance improvements X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Thu, Sep 24, 2020 at 06:02:48PM +0000, Ananyev, Konstantin wrote: > > > > > > Add minimum burst throughout the scheduler pipeline and a flush counter. > Replace ring API calls with local single threaded implementation where > possible. > > Signed-off-by: Radu Nicolau mailto:radu.nicolau@intel.com > > Thanks for the patch, a few comments inline. > > --- > drivers/event/sw/sw_evdev.h           | 11 +++- > drivers/event/sw/sw_evdev_scheduler.c | 83 > +++++++++++++++++++++++---- > 2 files changed, 81 insertions(+), 13 deletions(-) > > diff --git a/drivers/event/sw/sw_evdev.h b/drivers/event/sw/sw_evdev.h > index 7c77b2495..95e51065f 100644 > --- a/drivers/event/sw/sw_evdev.h > +++ b/drivers/event/sw/sw_evdev.h > @@ -29,7 +29,13 @@ > /* report dequeue burst sizes in buckets */  #define > SW_DEQ_STAT_BUCKET_SHIFT 2 > /* how many packets pulled from port by sched */ -#define > SCHED_DEQUEUE_BURST_SIZE 32 > +#define SCHED_DEQUEUE_BURST_SIZE 64 > + > +#define SCHED_MIN_BURST_SIZE 8 > +#define SCHED_NO_ENQ_CYCLE_FLUSH 256 > +/* set SCHED_DEQUEUE_BURST_SIZE to 64 or 128 when setting this to 1*/ > +#define SCHED_REFILL_ONCE_PER_CALL 1 > > Is it possible to make the above #define a runtime option? > Eg, --vdev event_sw,refill_iter=1 > > That would allow packaged versions of DPDK to be usable in both modes. > > + > > #define SW_PORT_HIST_LIST (MAX_SW_PROD_Q_DEPTH) /* size of our > history list */  #define NUM_SAMPLES 64 /* how many data points use > for average stats */ @@ -214,6 +220,9 @@ struct sw_evdev { >     uint32_t xstats_count_mode_port; >     uint32_t xstats_count_mode_queue; > > +    uint16_t sched_flush_count; > +    uint16_t sched_min_burst; > + >     /* Contains all ports - load balanced and directed */ >     struct sw_port ports[SW_PORTS_MAX] __rte_cache_aligned; > > diff --git a/drivers/event/sw/sw_evdev_scheduler.c > b/drivers/event/sw/sw_evdev_scheduler.c > index cff747da8..ca6d1caff 100644 > --- a/drivers/event/sw/sw_evdev_scheduler.c > +++ b/drivers/event/sw/sw_evdev_scheduler.c > @@ -26,6 +26,29 @@ > /* use cheap bit mixing, we only need to lose a few bits */  #define > SW_HASH_FLOWID(f) (((f) ^ (f >> 10)) & FLOWID_MASK) > > + > +/* single object enq and deq for non MT ring */ static > +__rte_always_inline void sw_nonmt_ring_dequeue(struct rte_ring *r, > +void **obj) { > +    if ((r->prod.tail - r->cons.tail) < 1) > +            return; > +    void **ring = (void **)&r[1]; > +    *obj = ring[r->cons.tail & r->mask]; > +    r->cons.tail++; > +} > +static __rte_always_inline int > +sw_nonmt_ring_enqueue(struct rte_ring *r, void *obj) { > +    if ((r->capacity + r->cons.tail - r->prod.tail) < 1) > +            return 0; > +    void **ring = (void **)&r[1]; > +    ring[r->prod.tail & r->mask] = obj; > +    r->prod.tail++; > +    return 1; > + > Why not make these APIs part of the rte_ring library? You could further optimize them by keeping the indices on the same cacheline. > I'm not sure there is any need for non thread-safe rings outside this particular case. > [Honnappa] I think if we add the APIs, we will find the use cases. > But, more than that, I understand that rte_ring structure is exposed to the application. The reason for doing that is the inline functions that rte_ring provides. IMO, we should still maintain modularity and should not use the internals of the rte_ring structure outside of the library. > > +1 to that. > > BTW, is there any real perf benefit from such micor-optimisation? I'd tend to view these as use-case specific, and I'm not sure we should clutter up the ring library with yet more functions, especially since they can't be mixed with the existing enqueue/dequeue functions, since they don't use the head pointers.