Re: [dpdk-dev] [EXT] Re: [PATCH v6 0/4] add IOVA = VA support in KNI

[Olivier Matz <olivier.matz@6wind.com>]

Hi,

On Tue, Jul 16, 2019 at 09:40:59AM +0000, Vamsi Krishna Attunuru wrote:
> 
> 
> > -----Original Message-----
> > From: Olivier Matz <olivier.matz@6wind.com>
> > Sent: Tuesday, July 16, 2019 2:17 PM
> > To: Burakov, Anatoly <anatoly.burakov@intel.com>
> > Cc: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Ferruh Yigit
> > <ferruh.yigit@intel.com>; Vamsi Krishna Attunuru <vattunuru@marvell.com>;
> > dev@dpdk.org; arybchenko@solarflare.com
> > Subject: Re: [dpdk-dev] [EXT] Re: [PATCH v6 0/4] add IOVA = VA support in KNI
> > 
> > Hi,
> > 
> > On Mon, Jul 15, 2019 at 10:38:53AM +0100, Burakov, Anatoly wrote:
> > > On 15-Jul-19 5:54 AM, Jerin Jacob Kollanukkaran wrote:
> > > > > > > > > > (also, i don't really like the name NO_PAGE_BOUND since
> > > > > > > > > > in memzone API there's a "bounded memzone" allocation
> > > > > > > > > > API, and this flag's name reads like objects would not
> > > > > > > > > > be bounded by page size, not that they won't cross page
> > > > > > > > > > boundary)
> > > > > > > > >
> > > > > > > > > No strong opinion for the name. What name you suggest?
> > > > > > > >
> > > > > > > > How about something like MEMPOOL_F_NO_PAGE_SPLIT?
> > > > > > >
> > > > > > > Looks good to me.
> > > > > > >
> > > > > > > In summary, Change wrt existing patch"
> > > > > > > - Change NO_PAGE_BOUND to MEMPOOL_F_NO_PAGE_SPLIT
> > > > > > > - Set this flag in  rte_pktmbuf_pool_create () when
> > > > > > rte_eal_has_hugepages() ||
> > > > > > >    rte_malloc_heap_socket_is_external(socket_id))
> > > > > >
> > > > > > If we are to have a special KNI allocation API, would we even need that?
> > > > >
> > > > > Not need this change in rte_pktmbuf_pool_create () if we introduce
> > > > > a new rte_kni_pktmbuf_pool_create () API.
> > > >
> > > > Ferruh, Olivier, Anatoly,
> > > >
> > > > Any objection to create new rte_kni_pktmbuf_pool_create () API to
> > > > embedded MEMPOOL_F_NO_PAGE_SPLIT flag requirement for KNI + IOVA
> > as
> > > > VA
> > > >
> > > >
> > >
> > > As long as we all are aware of what that means and agree with that
> > > consequence (namely, separate codepaths for KNI and other PMD's) then
> > > i have no specific objections.
> > 
> > Sorry for the late feedback.
> > 
> > I think we can change the default behavior of mempool populate(), to prevent
> > objects from being accross 2 pages, except if the size of the object is bigger than
> > the size of the page. This is already what is done in
> > rte_mempool_op_calc_mem_size_default() when we want to estimate the
> > amount of memory needed to allocate N objects.
> > 
> > This would avoid the introduction of a specific API to allocate packets for kni,
> > and a specific mempool flag.
> > 
> > About the problem of 9K mbuf mentionned by Anatoly, could we imagine a
> > check in kni code, that just returns an error "does not work with
> > size(mbuf) > size(page)" ?
> > 
> 
> Yes, change in default behavior avoids new APIs or flags.
> Two minor changes on top of  above suggestions.
> 1) Can flag(NO_PAGE_SPLIT) be retained.?,  sequence is like,  flag is set by default in rte_mempool_populate_default()
> and later it can be cleared based on obj_per_page in rte_mempool_op_calc_mem_size_default(). I do not see specific
> requirement of these flag apart from handling above sequence.

Sorry, I don't get why you want to keep this flag. Is it to facilitate
the error check in kni code?

The flags are used by the mempool user to ask for a specific behavior,
so if we change the default behavior, there is nothing to change to the
user API.

> 2) For problems of 9k mbuf, I think that check could be addressed in kni lib(in rte_kni_init and return error).

You can use rte_mempool_obj_iter() to iterate the objects (mbufs) in the
mempool, to ensure that none of them is accross 2 pages.