From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0AF1BA0528; Fri, 17 Jul 2020 19:09:29 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B16CD1BEBC; Fri, 17 Jul 2020 19:09:27 +0200 (CEST) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2053.outbound.protection.outlook.com [40.107.22.53]) by dpdk.org (Postfix) with ESMTP id 6F7D01BEB4 for ; Fri, 17 Jul 2020 19:09:26 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CKZI0wP4Qcl0b0XwC2AN3FKn5nUOj9hrp1HJTVeTfX+jQLUDV8oo/Br/QkNr7oyq4oZjQ2SKhID7gnReWCxZijICWNhibddE4wfg8GO3KLCv2vHQFLvNlEhucanCs6zc8lA+oAh+v6Enpq4rZefBTb5FfYPJ+1wPB55Z6egfJK6fZ1tMOSwqEDRgT+zEKKxyIaP7oEDOq/6858xm4ImoTqdJe+khrBCoVm/pk/2jUvpf4nNAMJMck4nsV88gXugB5pXVAATBECiic3xCBo5XRQ3B+r13iaYmBtIu8o+jV4XnK0v7APfaC3bj3oZQgPVwAGMdaiHY2wurSHPMfa5xNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=E/xgZKhGHKJ1zuIzM611ETGLhQCHamIbFAUOMKzgbLM=; b=gpiW4JhOkZAsqBKlho/Sz4YjdnMhCmIQo/LhQHg8wSKpbBtb/1VxBYIAT5lINIDqOZXA3jX+WhQB2vuOH85tDhC8NboZbaHJ5qXHecwlc6obFzsJ4XlWwPI+uYU+uy9rhZxLispBegq2hTREBWCyNHaKmvXDnF5I5HYjXZRmi9uSiXdd4xspKdak977piQRlr6stRqYIqIPIgc6+ivqavfhrFilvmukaVBFmSTermso2jHXtd4HvtphHd/rMn61q1X+cLvkZiABd/kdXYLEzp4q3OZs2pjz6R0asIB/dUPj4gDGyvZwOMw0/gSK3wlKDtVh4Ej6LVj12YHQJamXxcA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=E/xgZKhGHKJ1zuIzM611ETGLhQCHamIbFAUOMKzgbLM=; b=Wae7k7hNhMQcbFcG413CFCr+FliKN/42N6ppPsx/5zZM7CztFmnNEGGIAdxkaR8ZBZgHa16JA4Sc3UETi9E/mBCWiVrWycU9HAQl8+eWRCPyFPL18uoZS8rPU1cvjmNVI0nIJXEg6Wualn8wERpQq1Q4d0HusT1TwUywEvF7DTc= Received: from VI1PR05MB6718.eurprd05.prod.outlook.com (2603:10a6:800:13b::9) by VI1PR05MB6349.eurprd05.prod.outlook.com (2603:10a6:803:f6::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3174.20; Fri, 17 Jul 2020 17:09:23 +0000 Received: from VI1PR05MB6718.eurprd05.prod.outlook.com ([fe80::bc68:25c4:b452:f24d]) by VI1PR05MB6718.eurprd05.prod.outlook.com ([fe80::bc68:25c4:b452:f24d%9]) with mapi id 15.20.3174.027; Fri, 17 Jul 2020 17:09:23 +0000 From: Raslan Darawsheh To: Suanming Mou , Slava Ovsiienko , Matan Azrad CC: Ori Kam , "dev@dpdk.org" Thread-Topic: [PATCH v3 0/7] net/mlx5: add sys_mem_en devarg Thread-Index: AQHWXEFXSBCuCTUatEer9tAOYDym+KkMAW/A Date: Fri, 17 Jul 2020 17:09:22 +0000 Message-ID: References: <1594785603-152773-1-git-send-email-suanmingm@mellanox.com> <1594993865-396296-1-git-send-email-suanmingm@mellanox.com> In-Reply-To: <1594993865-396296-1-git-send-email-suanmingm@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: mellanox.com; dkim=none (message not signed) header.d=none;mellanox.com; dmarc=none action=none header.from=mellanox.com; x-originating-ip: [188.161.228.169] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: d7e64868-a9ed-46d1-0f57-08d82a7426fa x-ms-traffictypediagnostic: VI1PR05MB6349: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: mpcMB/jcpiFybW2LJiaeB1GeQfZreOGqAGinpUl6OR7UI7y2yy2IVUb+B1dCkynuDUHa7rWdlHWIugT2eFbClGMGgrOBXClQKiwcmb6D39fw5sox+q2hDXUXR5qjquDUuh3EV1+1EUwj49pKEpGw8vl5Ij/0fRj/+EvOYg/FTPMlL/P7PYP0g+r9LEeBL54xud0T6ge+yEnKEBf4+V3XDUkHH/OqvJ7dYq+EQQjM7Zu4pxkXIrSRpN8wdifzkmeWo04q43naRZyouNSJOzqy5VTJhe26Lo/8moJT4NQBQRWf/CqokdsJRQ0YdhMiH9HU x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR05MB6718.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(39860400002)(346002)(376002)(396003)(136003)(366004)(316002)(52536014)(4326008)(66556008)(66476007)(66946007)(66446008)(8936002)(6506007)(54906003)(53546011)(26005)(64756008)(33656002)(76116006)(55016002)(186003)(6636002)(4743002)(7696005)(5660300002)(71200400001)(86362001)(9686003)(478600001)(83380400001)(110136005)(2906002)(8676002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: bOGe8mtrx64ZuEix5WUmICgQeQK/Qp4NcSbLGGPxymo7bk1x2wOsl+6fpoPAsYDVtB1hzQvImukJ2bfB4RrMQ8VkbNMUE2txXSSrBNqXD7UV89f2hn2LePRMoMXemybuwMo4QubnMwlNTHKQEZJOzDxEZ/nSg4B5FAtVax0/a4hyPCyITQUoHXFQT61RNb6UBGyytqG98Dhck5SdB3yazmz9sXZLVOeBO7cOcRSGoc4lynCWWRR59+9+aNzuJ2kHZbV9LOVsWtVp2LH5saKkQ+uSBKyoQ3orJv7sNKAXztooa/rfriB6e1HV2D1GA1+jFDoYbgOXEYnkB5VS40WkWLGx/mq2Ob5JGn4Wdfd2XFPhJyH1U4N0z046kR2L991MQjmx4y0qHZeV0sD5M0+B9OQ8G8fCxYOMlWCbtqVH1Zgr8B8Yt8IT/bOYQo2PnMQQ8KNQnb0dQzZ/eURLd5ObSDlZzTIlnWtdLNjAQXKI648a3rgrKPf3cV2lSZkb/fP3 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR05MB6718.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: d7e64868-a9ed-46d1-0f57-08d82a7426fa X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Jul 2020 17:09:22.9598 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 3PDvdVZUDxYC6rxxbRhv2J/GuDbold7HxcekEAefdYr7+Sz1mLphIMAdofGnhVvqZ/reUPR4NHDFDB1XcCHMgA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR05MB6349 Subject: Re: [dpdk-dev] [PATCH v3 0/7] net/mlx5: add sys_mem_en devarg X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi > -----Original Message----- > From: Suanming Mou > Sent: Friday, July 17, 2020 4:51 PM > To: Slava Ovsiienko ; Matan Azrad > > Cc: Ori Kam ; Raslan Darawsheh > ; dev@dpdk.org > Subject: [PATCH v3 0/7] net/mlx5: add sys_mem_en devarg >=20 > Currently, for MLX5 PMD, once millions of flows created, the memory > consumption of the flows are also very huge. For the system with limited > memory, it means the system need to reserve most of the memory as huge > page memory to serve the flows in advance. And other normal applications > will have no chance to use this reserved memory any more. While most of > the time, the system will not have lots of flows, the reserved huge page > memory becomes a bit waste of memory at most of the time. >=20 > By the new sys_mem_en devarg, once set it to be true, it allows the PMD > allocate the memory from system by default with the new add mlx5 memory > management functions. Only once the MLX5_MEM_RTE flag is set, the > memory > will be allocate from rte, otherwise, it allocates memory from system. >=20 > So in this case, the system with limited memory no need to reserve most > of the memory for hugepage. Only some needed memory for datapath > objects > will be enough to allocated with explicitly flag. Other memory will be > allocated from system. For system with enough memory, no need to care > about the devarg, the memory will always be from rte hugepage. >=20 > One restriction is that for DPDK application with multiple PCI devices, > if the sys_mem_en devargs are different between the devices, the > sys_mem_en only gets the value from the first device devargs, and print > out a message to warn that. >=20 > --- >=20 > v3: > - Rebase on top of latest code. >=20 > v2: > - Add memory function call statistic. > - Change msl to aotmic. >=20 > --- >=20 > Suanming Mou (7): > common/mlx5: add mlx5 memory management functions > net/mlx5: add allocate memory from system devarg > net/mlx5: convert control path memory to unified malloc > common/mlx5: convert control path memory to unified malloc > common/mlx5: convert data path objects to unified malloc > net/mlx5: convert configuration objects to unified malloc > net/mlx5: convert Rx/Tx queue objects to unified malloc >=20 > doc/guides/nics/mlx5.rst | 7 + > drivers/common/mlx5/Makefile | 1 + > drivers/common/mlx5/linux/mlx5_glue.c | 13 +- > drivers/common/mlx5/linux/mlx5_nl.c | 5 +- > drivers/common/mlx5/meson.build | 1 + > drivers/common/mlx5/mlx5_common.c | 10 +- > drivers/common/mlx5/mlx5_common_mp.c | 7 +- > drivers/common/mlx5/mlx5_common_mr.c | 31 ++- > drivers/common/mlx5/mlx5_devx_cmds.c | 82 ++++--- > drivers/common/mlx5/mlx5_malloc.c | 306 > ++++++++++++++++++++++++ > drivers/common/mlx5/mlx5_malloc.h | 99 ++++++++ > drivers/common/mlx5/rte_common_mlx5_version.map | 6 + > drivers/net/mlx5/linux/mlx5_ethdev_os.c | 8 +- > drivers/net/mlx5/linux/mlx5_os.c | 28 ++- > drivers/net/mlx5/mlx5.c | 108 +++++---- > drivers/net/mlx5/mlx5.h | 1 + > drivers/net/mlx5/mlx5_ethdev.c | 15 +- > drivers/net/mlx5/mlx5_flow.c | 45 ++-- > drivers/net/mlx5/mlx5_flow_dv.c | 46 ++-- > drivers/net/mlx5/mlx5_flow_meter.c | 11 +- > drivers/net/mlx5/mlx5_flow_verbs.c | 8 +- > drivers/net/mlx5/mlx5_mp.c | 3 +- > drivers/net/mlx5/mlx5_rss.c | 13 +- > drivers/net/mlx5/mlx5_rxq.c | 74 +++--- > drivers/net/mlx5/mlx5_txpp.c | 30 +-- > drivers/net/mlx5/mlx5_txq.c | 82 +++---- > drivers/net/mlx5/mlx5_utils.c | 60 +++-- > drivers/net/mlx5/mlx5_utils.h | 2 +- > drivers/net/mlx5/mlx5_vlan.c | 8 +- > 29 files changed, 797 insertions(+), 313 deletions(-) > create mode 100644 drivers/common/mlx5/mlx5_malloc.c > create mode 100644 drivers/common/mlx5/mlx5_malloc.h >=20 > -- > 1.8.3.1 Series applied to next-net-mlx, Kindest regards, Raslan Darawsheh