Re: [dpdk-dev] Accessing 2nd cacheline in rte_pktmbuf_prefree_seg()

["Ananyev, Konstantin" <konstantin.ananyev@intel.com>]

Hi Yongseok,

> > On Feb 13, 2018, at 2:45 PM, Yongseok Koh <yskoh@mellanox.com> wrote:
> >
> > Hi Olivier
> >
> > I'm wondering why rte_pktmbuf_prefree_seg() checks m->next instead of
> > m->nb_segs? As 'next' is in the 2nd cacheline, checking nb_segs seems beneficial
> > to the cases where almost mbufs have single segment.
> >
> > A customer reported high rate of cache misses in the code and I thought the
> > following patch could be helpful. I haven't had them try it yet but just wanted
> > to hear from you.
> >
> > I'd appreciate if you can review this idea.
> >
> > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> > index 62740254d..96edbcb9e 100644
> > --- a/lib/librte_mbuf/rte_mbuf.h
> > +++ b/lib/librte_mbuf/rte_mbuf.h
> > @@ -1398,7 +1398,7 @@ rte_pktmbuf_prefree_seg(struct rte_mbuf *m)
> >                if (RTE_MBUF_INDIRECT(m))
> >                        rte_pktmbuf_detach(m);
> >
> > -               if (m->next != NULL) {
> > +               if (m->nb_segs > 1) {
> >                        m->next = NULL;
> >                        m->nb_segs = 1;
> >                }
> > @@ -1410,7 +1410,7 @@ rte_pktmbuf_prefree_seg(struct rte_mbuf *m)
> >                if (RTE_MBUF_INDIRECT(m))
> >                        rte_pktmbuf_detach(m);
> >
> > -               if (m->next != NULL) {
> > +               if (m->nb_segs > 1) {
> >                        m->next = NULL;
> >                        m->nb_segs = 1;
> >                }
> 
> Well, m->pool in the 2nd cacheline has to be accessed anyway in order to put it back to the mempool.
> It looks like the cache miss is unavoidable.

As a thought: in theory PMD can store pool pointer together with each mbuf it has to free,
then it could be something like:

if (rte_pktmbuf_prefree_seg(m[x] != NULL)
   rte_mempool_put(pool[x], m[x]);

Then what you suggested above might help.
Konstantin