Re: [dpdk-dev] [PATCH v2 2/2] mbuf: add bulk free function

[Bruce Richardson <bruce.richardson@intel.com>]

On Thu, Sep 26, 2019 at 10:11:06PM +0200, Mattias Rönnblom wrote:
> On 2019-09-26 10:30, Bruce Richardson wrote:
> > On Wed, Sep 25, 2019 at 09:02:28PM +0200, Mattias Rönnblom wrote:
> > > On 2019-09-25 14:03, Morten Brørup wrote:
> > > > Add function for freeing a bulk of mbufs.
> > > > 
> > > > Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
> > > > ---
> > > >    lib/librte_mbuf/rte_mbuf.c | 35 +++++++++++++++++++++++++++++++++++
> > > >    lib/librte_mbuf/rte_mbuf.h | 16 +++++-----------
> > > >    2 files changed, 40 insertions(+), 11 deletions(-)
> > > > 
> > > > diff --git a/lib/librte_mbuf/rte_mbuf.c b/lib/librte_mbuf/rte_mbuf.c
> > > > index 37718d49c..b63a0eced 100644
> > > > --- a/lib/librte_mbuf/rte_mbuf.c
> > > > +++ b/lib/librte_mbuf/rte_mbuf.c
> > > > @@ -245,6 +245,41 @@ int rte_mbuf_check(const struct rte_mbuf *m, int is_header,
> > > >    	return 0;
> > > >    }
> > > > +/**
> > > > + * Maximum bulk of mbufs rte_pktmbuf_free_bulk() returns to mempool.
> > > > + */
> > > > +#define RTE_PKTMBUF_FREE_BULK_SZ 64
> > > > +
> > > > +/* Free a bulk of mbufs back into their original mempools. */
> > > > +void rte_pktmbuf_free_bulk(struct rte_mbuf **mbufs, unsigned int count)
> > > > +{
> > > > +	struct rte_mbuf *m, *free[RTE_PKTMBUF_FREE_BULK_SZ];
> > > > +	unsigned int idx, nb_free = 0;
> > > > +
> > > > +	for (idx = 0; idx < count; idx++) {
> > > > +		m = mbufs[idx];
> > > > +		if (unlikely(m == NULL))
> > > > +			continue;
> > > > +
> > > > +		__rte_mbuf_sanity_check(m, 1);
> > > > +		m = rte_pktmbuf_prefree_seg(m);
> > > > +		if (unlikely(m == NULL))
> > > > +			continue;
> > > > +
> > > > +		if (nb_free >= RTE_PKTMBUF_FREE_BULK_SZ ||
> > > > +		    (nb_free > 0 && m->pool != free[0]->pool)) {
> > > 
> > > Maybe an unlikely() would be in order here?
> > > 
> > I'd caution against it, since it can penalize the cold branch a lot. If a
> > branch really is predictable the HW branch predictors generally are good
> > enough to handle it at runtime. So long as a path is a valid path for a
> > runtime app, i.e. not something like a fatal error only ever hit once in an
> > entire run, I'd tend to omit likely()/unlikely() calls unless profiling
> > shows a real performance difference.
> > 
> 
> Let's see if I understand you: your worry is that wrapping that expression
> in an unlikely() will lead to code that is slower (than w/o the hint), if
> during runtime the probability turns out to be 50/50?
> 
While not an expert, I believe that the use of likely/unlikely can cause the
unexpected part of the branch to be moved to a different part of the code
and potentially be more expensive to call, meaning that the performance may be
poorer even if the probability is lower than 50/50.

> Wouldn't leaving out unlikely() just lead to the compiler using its fancy
> heuristics in an attempt to come to a conclusion, what path is the more
> likely?
> 
> About HW branch prediction - I'm sure it's good, but still the compiler
> needs to decided which code code path requires a branch, and which need not.
> Even if HW branch prediction successfully predicted a branch being taken,
> actually branching is going to be somewhat more expensive than to not
> branch?

The cost difference between a taken and untaken branch should be
unnoticable so long as the branch is correctly predicted - which if does
always go one way, it will be each time each time after the first. Overall,
though, I suspect the presence of likely/unlikely is going to make any real
difference, so I'd therefore err on the side of leaving it out in the
absense of evidence that it helps in some cases.

/Bruce