DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [RFC] mempool: add non-IO flag
@ 2021-08-12 12:43 Dmitry Kozlyuk
  2021-08-25  8:01 ` Thomas Monjalon
  0 siblings, 1 reply; 3+ messages in thread
From: Dmitry Kozlyuk @ 2021-08-12 12:43 UTC (permalink / raw)
  To: dev
  Cc: Matan Azrad, Olivier Matz, Andrew Rybchenko, Slava Ovsiienko,
	Thomas Monjalon

We propose to add a mempool flag MEMPOOL_F_NON_IO to mark pools of objects that
will not be used with device IO and their memory for DMA.  This will allow
saving IOMMU entries by not mapping the memory used by such pools.

Immediate use case is MLX5 PMD.  The hardware has its internal IOMMU where PMD
registers the memory.  On the data path, PMD translates VA into a key consumed
by the device IOMMU.  It is impractical for the PMD to register all allocated
memory because of increased lookup cost both in HW and SW.  Most often mbuf
memory comes from mempools, so if PMD tracked them, it could almost always have
mbuf memory registered before an mbuf hits the PMD. The new flag would prevent
the PMD for registering memory that will never need it. Tracking the mempools
and dealing with them in MLX5 PMD is the next step after the proposed change.

A possible use case is IOMMU management in EAL.  Mempool could translate the new
flag to a hint to the memory manager, which would use it to skip adding IOMMU
entries in some cases.

It was considered to add MEMPOOL_F_IO with the opposite meaning. It would be
automatically set for pktmbuf pools; user would be able to set it for other
pools.  However, current assumption is that all DPDK memory is DMA-able,
it is controversial to have a flag asserting this fact.

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [dpdk-dev] [RFC] mempool: add non-IO flag
  2021-08-12 12:43 [dpdk-dev] [RFC] mempool: add non-IO flag Dmitry Kozlyuk
@ 2021-08-25  8:01 ` Thomas Monjalon
  2021-08-25 17:28   ` Ajit Khaparde
  0 siblings, 1 reply; 3+ messages in thread
From: Thomas Monjalon @ 2021-08-25  8:01 UTC (permalink / raw)
  To: Dmitry Kozlyuk
  Cc: dev, Matan Azrad, Olivier Matz, Andrew Rybchenko,
	Slava Ovsiienko, david.marchand, maxime.coquelin,
	bruce.richardson, ferruh.yigit, anatoly.burakov, jerinj,
	honnappa.nagarahalli, ajit.khaparde

+1, I support this idea.

12/08/2021 14:43, Dmitry Kozlyuk:
> We propose to add a mempool flag MEMPOOL_F_NON_IO to mark pools of objects that
> will not be used with device IO and their memory for DMA.  This will allow
> saving IOMMU entries by not mapping the memory used by such pools.
> 
> Immediate use case is MLX5 PMD.  The hardware has its internal IOMMU where PMD
> registers the memory.  On the data path, PMD translates VA into a key consumed
> by the device IOMMU.  It is impractical for the PMD to register all allocated
> memory because of increased lookup cost both in HW and SW.  Most often mbuf
> memory comes from mempools, so if PMD tracked them, it could almost always have
> mbuf memory registered before an mbuf hits the PMD. The new flag would prevent
> the PMD for registering memory that will never need it. Tracking the mempools
> and dealing with them in MLX5 PMD is the next step after the proposed change.
> 
> A possible use case is IOMMU management in EAL.  Mempool could translate the new
> flag to a hint to the memory manager, which would use it to skip adding IOMMU
> entries in some cases.
> 
> It was considered to add MEMPOOL_F_IO with the opposite meaning. It would be
> automatically set for pktmbuf pools; user would be able to set it for other
> pools.  However, current assumption is that all DPDK memory is DMA-able,
> it is controversial to have a flag asserting this fact.



^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [dpdk-dev] [RFC] mempool: add non-IO flag
  2021-08-25  8:01 ` Thomas Monjalon
@ 2021-08-25 17:28   ` Ajit Khaparde
  0 siblings, 0 replies; 3+ messages in thread
From: Ajit Khaparde @ 2021-08-25 17:28 UTC (permalink / raw)
  To: Thomas Monjalon
  Cc: Dmitry Kozlyuk, dev, Matan Azrad, Olivier Matz, Andrew Rybchenko,
	Slava Ovsiienko, David Marchand, Maxime Coquelin,
	Bruce Richardson, Ferruh Yigit, Anatoly Burakov,
	Jerin Jacob Kollanukkaran, Honnappa Nagarahalli

[-- Attachment #1: Type: text/plain, Size: 1553 bytes --]

On Wed, Aug 25, 2021 at 1:01 AM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> +1, I support this idea.
>
> 12/08/2021 14:43, Dmitry Kozlyuk:
> > We propose to add a mempool flag MEMPOOL_F_NON_IO to mark pools of objects that
> > will not be used with device IO and their memory for DMA.  This will allow
> > saving IOMMU entries by not mapping the memory used by such pools.
+1

> >
> > Immediate use case is MLX5 PMD.  The hardware has its internal IOMMU where PMD
> > registers the memory.  On the data path, PMD translates VA into a key consumed
> > by the device IOMMU.  It is impractical for the PMD to register all allocated
> > memory because of increased lookup cost both in HW and SW.  Most often mbuf
> > memory comes from mempools, so if PMD tracked them, it could almost always have
> > mbuf memory registered before an mbuf hits the PMD. The new flag would prevent
> > the PMD for registering memory that will never need it. Tracking the mempools
> > and dealing with them in MLX5 PMD is the next step after the proposed change.
> >
> > A possible use case is IOMMU management in EAL.  Mempool could translate the new
> > flag to a hint to the memory manager, which would use it to skip adding IOMMU
> > entries in some cases.
> >
> > It was considered to add MEMPOOL_F_IO with the opposite meaning. It would be
> > automatically set for pktmbuf pools; user would be able to set it for other
> > pools.  However, current assumption is that all DPDK memory is DMA-able,
> > it is controversial to have a flag asserting this fact.
+1

>
>

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2021-08-25 17:28 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-12 12:43 [dpdk-dev] [RFC] mempool: add non-IO flag Dmitry Kozlyuk
2021-08-25  8:01 ` Thomas Monjalon
2021-08-25 17:28   ` Ajit Khaparde

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).