On Sun, Nov 5, 2023 at 5:56 PM Avi Kivity <avi@scylladb.com> wrote:

Thanks, it makes sense. I'll get around to it "eventually".

On Thu, 2023-11-02 at 11:04 +0100, Thomas Monjalon wrote:
Hello,

While looking at Seastar, I see it uses this patch on top of DPDK:

        build: add meson options of max_memseg_lists

        RTE_MAX_MEMSEG_LISTS = 128 is not enough for high-memory machines,
        in our case, we need to increase it to 8192.
        so add an option so user can override it.

        https://github.com/scylladb/dpdk/commit/cafaa3cf457584de

I think we could allow to configure this at runtime,
as we did already for RTE_MAX_MEMZONE:
we've added rte_memzone_max_set() / rte_memzone_max_get().

Opinions, comments, volunteers?

Hi Thomas,

Thank you for looking into it. I sent this patch[0] to DPDK 2 years ago. but i failed to find a solid proof to prove that we need such a massive number, and failed to follow-up on the suggestion[1] on calculating his number based on the lcores / numa node as I was trying to port the newer dpdk to seastar at that moment, so dropped the ball on my end, sorry for that. just revisited the places where we use RTE_MAX_MEMSEG_LISTS. it seems it would be a bigger effort to make it a run-time configurable option instead of a compile-time one.

[0] https://inbox.dpdk.org/dev/20211013205417.84119-3-tchaikov@gmail.com/

[1] https://inbox.dpdk.org/dev/2642296.XfZ1dg20Xv@thomas/

Regards
Kefu Chai