DPDK patches and discussions
 help / color / mirror / Atom feed
From: Suanming Mou <suanmingm@mellanox.com>
To: viacheslavo@mellanox.com, matan@mellanox.com
Cc: orika@mellanox.com, rasland@mellanox.com, dev@dpdk.org
Subject: [dpdk-dev] [PATCH v2 0/7] net/mlx5: add sys_mem_en devarg
Date: Thu, 16 Jul 2020 17:20:09 +0800
Message-ID: <1594891216-11778-1-git-send-email-suanmingm@mellanox.com> (raw)
In-Reply-To: <1594785603-152773-1-git-send-email-suanmingm@mellanox.com>

Currently, for MLX5 PMD, once millions of flows created, the memory
consumption of the flows are also very huge. For the system with limited
memory, it means the system need to reserve most of the memory as huge
page memory to serve the flows in advance. And other normal applications
will have no chance to use this reserved memory any more. While most of
the time, the system will not have lots of flows, the  reserved huge page
memory becomes a bit waste of memory at most of the time.

By the new sys_mem_en devarg, once set it to be true, it allows the PMD
allocate the memory from system by default with the new add mlx5 memory
management functions. Only once the MLX5_MEM_RTE flag is set, the memory
will be allocate from rte, otherwise, it allocates memory from system.

So in this case, the system with limited memory no need to reserve most
of the memory for hugepage. Only some needed memory for datapath objects
will be enough to allocated with explicitly flag. Other memory will be
allocated from system. For system with enough memory, no need to care
about the devarg, the memory will always be from rte hugepage.

One restriction is that for DPDK application with multiple PCI devices,
if the sys_mem_en devargs are different between the devices, the
sys_mem_en only gets the value from the first device devargs, and print
out a message to warn that.

---

v2:
 - Add memory function call statistic.
 - Change msl to aotmic.

---

Suanming Mou (7):
  common/mlx5: add mlx5 memory management functions
  net/mlx5: add allocate memory from system devarg
  net/mlx5: convert control path memory to unified malloc
  common/mlx5: convert control path memory to unified malloc
  common/mlx5: convert data path objects to unified malloc
  net/mlx5: convert configuration objects to unified malloc
  net/mlx5: convert Rx/Tx queue objects to unified malloc

 doc/guides/nics/mlx5.rst                        |   7 +
 drivers/common/mlx5/Makefile                    |   1 +
 drivers/common/mlx5/linux/mlx5_glue.c           |  13 +-
 drivers/common/mlx5/linux/mlx5_nl.c             |   5 +-
 drivers/common/mlx5/meson.build                 |   1 +
 drivers/common/mlx5/mlx5_common.c               |  10 +-
 drivers/common/mlx5/mlx5_common_mp.c            |   7 +-
 drivers/common/mlx5/mlx5_common_mr.c            |  31 ++-
 drivers/common/mlx5/mlx5_devx_cmds.c            |  75 +++---
 drivers/common/mlx5/mlx5_malloc.c               | 306 ++++++++++++++++++++++++
 drivers/common/mlx5/mlx5_malloc.h               |  99 ++++++++
 drivers/common/mlx5/rte_common_mlx5_version.map |   6 +
 drivers/net/mlx5/linux/mlx5_ethdev_os.c         |   8 +-
 drivers/net/mlx5/linux/mlx5_os.c                |  28 ++-
 drivers/net/mlx5/mlx5.c                         | 108 +++++----
 drivers/net/mlx5/mlx5.h                         |   1 +
 drivers/net/mlx5/mlx5_ethdev.c                  |  15 +-
 drivers/net/mlx5/mlx5_flow.c                    |  45 ++--
 drivers/net/mlx5/mlx5_flow_dv.c                 |  46 ++--
 drivers/net/mlx5/mlx5_flow_meter.c              |  11 +-
 drivers/net/mlx5/mlx5_flow_verbs.c              |   8 +-
 drivers/net/mlx5/mlx5_mp.c                      |   3 +-
 drivers/net/mlx5/mlx5_rss.c                     |  13 +-
 drivers/net/mlx5/mlx5_rxq.c                     |  74 +++---
 drivers/net/mlx5/mlx5_txq.c                     |  44 ++--
 drivers/net/mlx5/mlx5_utils.c                   |  60 +++--
 drivers/net/mlx5/mlx5_utils.h                   |   2 +-
 drivers/net/mlx5/mlx5_vlan.c                    |   8 +-
 28 files changed, 759 insertions(+), 276 deletions(-)
 create mode 100644 drivers/common/mlx5/mlx5_malloc.c
 create mode 100644 drivers/common/mlx5/mlx5_malloc.h

-- 
1.8.3.1


  parent reply	other threads:[~2020-07-16  9:20 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-15  3:59 [dpdk-dev] [PATCH " Suanming Mou
2020-07-15  3:59 ` [dpdk-dev] [PATCH 1/7] common/mlx5: add mlx5 memory management functions Suanming Mou
2020-07-15  3:59 ` [dpdk-dev] [PATCH 2/7] net/mlx5: add allocate memory from system devarg Suanming Mou
2020-07-15  3:59 ` [dpdk-dev] [PATCH 3/7] net/mlx5: convert control path memory to unified malloc Suanming Mou
2020-07-15  4:00 ` [dpdk-dev] [PATCH 4/7] common/mlx5: " Suanming Mou
2020-07-15  4:00 ` [dpdk-dev] [PATCH 5/7] common/mlx5: convert data path objects " Suanming Mou
2020-07-15  4:00 ` [dpdk-dev] [PATCH 6/7] net/mlx5: convert configuration " Suanming Mou
2020-07-15  4:00 ` [dpdk-dev] [PATCH 7/7] net/mlx5: convert Rx/Tx queue " Suanming Mou
2020-07-16  9:20 ` Suanming Mou [this message]
2020-07-16  9:20   ` [dpdk-dev] [PATCH v2 1/7] common/mlx5: add mlx5 memory management functions Suanming Mou
2020-07-16  9:20   ` [dpdk-dev] [PATCH v2 2/7] net/mlx5: add allocate memory from system devarg Suanming Mou
2020-07-16  9:20   ` [dpdk-dev] [PATCH v2 3/7] net/mlx5: convert control path memory to unified malloc Suanming Mou
2020-07-16  9:20   ` [dpdk-dev] [PATCH v2 4/7] common/mlx5: " Suanming Mou
2020-07-16  9:20   ` [dpdk-dev] [PATCH v2 5/7] common/mlx5: convert data path objects " Suanming Mou
2020-07-16  9:20   ` [dpdk-dev] [PATCH v2 6/7] net/mlx5: convert configuration " Suanming Mou
2020-07-16  9:20   ` [dpdk-dev] [PATCH v2 7/7] net/mlx5: convert Rx/Tx queue " Suanming Mou
2020-07-17 13:50 ` [dpdk-dev] [PATCH v3 0/7] net/mlx5: add sys_mem_en devarg Suanming Mou
2020-07-17 13:50   ` [dpdk-dev] [PATCH v3 1/7] common/mlx5: add mlx5 memory management functions Suanming Mou
2020-07-17 13:51   ` [dpdk-dev] [PATCH v3 2/7] net/mlx5: add allocate memory from system devarg Suanming Mou
2020-07-17 13:51   ` [dpdk-dev] [PATCH v3 3/7] net/mlx5: convert control path memory to unified malloc Suanming Mou
2020-07-17 13:51   ` [dpdk-dev] [PATCH v3 4/7] common/mlx5: " Suanming Mou
2020-07-17 13:51   ` [dpdk-dev] [PATCH v3 5/7] common/mlx5: convert data path objects " Suanming Mou
2020-07-17 13:51   ` [dpdk-dev] [PATCH v3 6/7] net/mlx5: convert configuration " Suanming Mou
2020-07-17 13:51   ` [dpdk-dev] [PATCH v3 7/7] net/mlx5: convert Rx/Tx queue " Suanming Mou
2020-07-17 17:09   ` [dpdk-dev] [PATCH v3 0/7] net/mlx5: add sys_mem_en devarg Raslan Darawsheh

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1594891216-11778-1-git-send-email-suanmingm@mellanox.com \
    --to=suanmingm@mellanox.com \
    --cc=dev@dpdk.org \
    --cc=matan@mellanox.com \
    --cc=orika@mellanox.com \
    --cc=rasland@mellanox.com \
    --cc=viacheslavo@mellanox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git