DPDK patches and discussions
 help / color / mirror / Atom feed
From: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
To: Amit Prakash Shukla <amitprakashs@marvell.com>
Cc: David Marchand <david.marchand@redhat.com>,
	Thomas Monjalon <thomas@monjalon.net>,
	Anatoly Burakov <anatoly.burakov@intel.com>,
	"dev@dpdk.org" <dev@dpdk.org>,
	Jerin Jacob Kollanukkaran <jerinj@marvell.com>,
	"bruce.richardson@intel.com" <bruce.richardson@intel.com>,
	"ciara.power@intel.com" <ciara.power@intel.com>
Subject: Re: [EXT] Re: [PATCH v5 2/2] mem: telemetry support for system memory information
Date: Thu, 20 Oct 2022 22:18:27 +0300	[thread overview]
Message-ID: <20221020221827.73275090@sovereign> (raw)
In-Reply-To: <PH0PR18MB516754E147E061C18D3B0A33C8239@PH0PR18MB5167.namprd18.prod.outlook.com>

2022-10-11 07:10 (UTC+0000), Amit Prakash Shukla:
> Thanks David for the feedback. Please find the proposed fixed inline.
> 
> > -----Original Message-----
> > From: David Marchand <david.marchand@redhat.com>
> > Sent: Saturday, October 8, 2022 1:17 AM
> > To: Amit Prakash Shukla <amitprakashs@marvell.com>; Thomas Monjalon
> > <thomas@monjalon.net>; Anatoly Burakov <anatoly.burakov@intel.com>;
> > dmitry.kozliuk@gmail.com
> > Cc: dev@dpdk.org; Jerin Jacob Kollanukkaran <jerinj@marvell.com>;
> > bruce.richardson@intel.com; ciara.power@intel.com
> > Subject: [EXT] Re: [PATCH v5 2/2] mem: telemetry support for system
> > memory information
> > 
> > External Email
> > 
> > ----------------------------------------------------------------------
> > On Thu, Sep 29, 2022 at 1:44 PM Amit Prakash Shukla
> > <amitprakashs@marvell.com> wrote:  
> > >
> > > Changes adds telemetry support to display system memory information,
> > > allocated using calls malloc, calloc, mmap, etc. This patch is based
> > > on malloc_info. This patch adds following endpoints:  
> > 
> > malloc_info is a GNU extension.
> > It is not available in musl and it breaks compilation in Alpine Linux.
> > So I can't take this patch.  
> 
> Shall we limit this for Glibc at compile time ?
> 
> +#if defined __GLIBC__ && defined __GLIBC_PREREQ
> +#if __GLIBC_PREREQ(2, 10)
>         /* Gets system memory stat's XML format. */
>         ret = malloc_info(0, fp);
> +#endif
> +#else
> +       RTE_LOG(DEBUG, EAL, "Error: malloc_info not supported by libc\n");
> +       fclose(fp);
> +       return -1;
> +#endif
> 
> I will send out a v6 if the above solution is fine.
> 
> > 
> >   
> > >
> > > 1. /sysmem/sys_heap_list
> > > The commands displays the arenas currently in use.
> > > Example:  
> > > --> /sysmem/sys_heap_list  
> > > {"/sysmem/sys_heap_list": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]}
> > >  
> > 
> > I am unsure about the command names.
> > 
> > Those commands are really low level and tied to glibc malloc.
> > It is unlikely we will have an unified naming with other libc/OS.
> > 
> > I would prefer another pair of eyes, especially on this patch.
> > Dmitry, Anatoly?

I agree with David.

Microsoft CRT provides completely different statistics.
I can't say for FreeBSD.
Even if GNU libc is used, the process can be run with jemalloc, for example,
and this statistics become meaningless then.
What is "system" memory, anyway? It can be acquired bypassing malloc,
it can be hugepages mmap'd without DPDK, etc.

What is the task this API must solve?
My guess is that monitoring needs to know how much physical memory
1) the process uses and 2) how much it can use, so on Linux this API
could be implemented using getrusage() and getrlimit().
I don't think that a fully portable implementation is possible,
but at least API will be unified.

  reply	other threads:[~2022-10-20 19:18 UTC|newest]

Thread overview: 42+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-05-19  6:30 [PATCH] mem: telemetry support for memseg and element information Amit Prakash Shukla
2022-05-19 12:42 ` David Marchand
2022-05-19 18:57 ` [PATCH v2] " Amit Prakash Shukla
2022-05-23 11:14   ` Bruce Richardson
2022-05-23 13:35     ` [EXT] " Amit Prakash Shukla
2022-05-23 13:43       ` Bruce Richardson
2022-05-24 10:30         ` Amit Prakash Shukla
2022-05-24 10:33 ` [PATCH v3] " Amit Prakash Shukla
2022-05-25 10:33 ` [PATCH v4 1/2] " Amit Prakash Shukla
2022-05-25 10:33   ` [PATCH v4 2/2] mem: telemetry support for system memory information Amit Prakash Shukla
2022-06-30  5:54     ` Amit Prakash Shukla
2022-07-21 11:21       ` Amit Prakash Shukla
2022-06-14 12:50   ` [PATCH v4 1/2] mem: telemetry support for memseg and element information Amit Prakash Shukla
2022-06-30  5:52     ` Amit Prakash Shukla
2022-07-21 11:20       ` Amit Prakash Shukla
2022-09-29  8:29   ` David Marchand
2022-09-29 11:30     ` [EXT] " Amit Prakash Shukla
2022-09-29 11:43   ` [PATCH v5 " Amit Prakash Shukla
2022-09-29 11:43     ` [PATCH v5 2/2] mem: telemetry support for system memory information Amit Prakash Shukla
2022-10-07 19:46       ` David Marchand
2022-10-11  7:10         ` [EXT] " Amit Prakash Shukla
2022-10-20 19:18           ` Dmitry Kozlyuk [this message]
2022-10-20 19:50             ` Stephen Hemminger
2022-10-06  7:07     ` [PATCH v5 1/2] mem: telemetry support for memseg and element information Amit Prakash Shukla
2022-10-07 19:52       ` David Marchand
2022-10-07 19:48     ` David Marchand
2022-10-11  7:22       ` [EXT] " Amit Prakash Shukla
2022-10-20 11:40     ` Dmitry Kozlyuk
2022-10-21 19:26       ` [EXT] " Amit Prakash Shukla
2022-10-21 20:07         ` Dmitry Kozlyuk
2022-10-25  7:25           ` Amit Prakash Shukla
2022-10-25 11:51     ` [PATCH v6] " Amit Prakash Shukla
2022-10-25 13:02       ` [PATCH v7] " Amit Prakash Shukla
2022-12-06 11:46         ` Amit Prakash Shukla
2023-01-30 10:18           ` Amit Prakash Shukla
2023-02-20 11:10         ` Thomas Monjalon
2023-02-28  7:30           ` [EXT] " Amit Prakash Shukla
2023-05-15 11:51             ` Amit Prakash Shukla
2023-05-16 10:47         ` Burakov, Anatoly
2023-05-17  9:08           ` [EXT] " Amit Prakash Shukla
2023-05-17  9:21         ` [PATCH v8] " Amit Prakash Shukla
2023-06-07 20:40           ` David Marchand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20221020221827.73275090@sovereign \
    --to=dmitry.kozliuk@gmail.com \
    --cc=amitprakashs@marvell.com \
    --cc=anatoly.burakov@intel.com \
    --cc=bruce.richardson@intel.com \
    --cc=ciara.power@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=jerinj@marvell.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).