configuration of memseg lists number

configuration of memseg lists number

[Thomas Monjalon]

Hello,

While looking at Seastar, I see it uses this patch on top of DPDK:

	build: add meson options of max_memseg_lists

	RTE_MAX_MEMSEG_LISTS = 128 is not enough for high-memory machines,
	in our case, we need to increase it to 8192.
	so add an option so user can override it.

	https://github.com/scylladb/dpdk/commit/cafaa3cf457584de

I think we could allow to configure this at runtime,
as we did already for RTE_MAX_MEMZONE:
we've added rte_memzone_max_set() / rte_memzone_max_get().

Opinions, comments, volunteers?

Re: configuration of memseg lists number

[Avi Kivity]

[-- Attachment #1: Type: text/plain, Size: 785 bytes --]

Thanks, it makes sense. I'll get around to it "eventually".

On Thu, 2023-11-02 at 11:04 +0100, Thomas Monjalon wrote:
> Hello,
> 
> While looking at Seastar, I see it uses this patch on top of DPDK:
> 
>         build: add meson options of max_memseg_lists
> 
>         RTE_MAX_MEMSEG_LISTS = 128 is not enough for high-memory
> machines,
>         in our case, we need to increase it to 8192.
>         so add an option so user can override it.
> 
>         https://github.com/scylladb/dpdk/commit/cafaa3cf457584de
> 
> I think we could allow to configure this at runtime,
> as we did already for RTE_MAX_MEMZONE:
> we've added rte_memzone_max_set() / rte_memzone_max_get().
> 
> Opinions, comments, volunteers?
> 
> 


[-- Attachment #2: Type: text/html, Size: 1682 bytes --]

Re: configuration of memseg lists number

[kefu chai]

[-- Attachment #1: Type: text/plain, Size: 1542 bytes --]

On Sun, Nov 5, 2023 at 5:56 PM Avi Kivity <avi@scylladb.com> wrote:

> Thanks, it makes sense. I'll get around to it "eventually".
>
> On Thu, 2023-11-02 at 11:04 +0100, Thomas Monjalon wrote:
>
> Hello,
>
> While looking at Seastar, I see it uses this patch on top of DPDK:
>
>         build: add meson options of max_memseg_lists
>
>         RTE_MAX_MEMSEG_LISTS = 128 is not enough for high-memory machines,
>         in our case, we need to increase it to 8192.
>         so add an option so user can override it.
>
>         https://github.com/scylladb/dpdk/commit/cafaa3cf457584de
>
> I think we could allow to configure this at runtime,
> as we did already for RTE_MAX_MEMZONE:
> we've added rte_memzone_max_set() / rte_memzone_max_get().
>
> Opinions, comments, volunteers?
>
>
Hi Thomas,

Thank you for looking into it. I sent this patch[0] to DPDK 2 years ago.
but i failed to find a solid proof to prove that we need such a massive
number, and failed to follow-up on the suggestion[1] on calculating his
number based on the lcores / numa node as I was trying to port the newer
dpdk to seastar at that moment, so dropped the ball on my end, sorry for
that. just revisited the places where we use RTE_MAX_MEMSEG_LISTS. it seems
it would be a bigger effort to make it a run-time configurable option
instead of a compile-time one.

--
[0] https://inbox.dpdk.org/dev/20211013205417.84119-3-tchaikov@gmail.com/
[1] https://inbox.dpdk.org/dev/2642296.XfZ1dg20Xv@thomas/

-- 
Regards
Kefu Chai

[-- Attachment #2: Type: text/html, Size: 2722 bytes --]