From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 00A16A034E; Fri, 5 Jun 2020 05:07:19 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 434A51D619; Fri, 5 Jun 2020 05:07:19 +0200 (CEST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2060.outbound.protection.outlook.com [40.107.21.60]) by dpdk.org (Postfix) with ESMTP id 4A5B61D616 for ; Fri, 5 Jun 2020 05:07:18 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IlAGeB5rZH5+K6I4LoM+4pTUBfAtpAsDdrFgHL9NOc+qz1lj3f5ZkWlpNFLQd+L14JZZwqaX6mMojJKZwDkAb6eqwIlfPz8v41v0nPRMRW8eOOQiy3orlr8ASL8xsKBzcZq2cEFzIYAlKxPD3wsG/10a9tA8oAGpg1iVe/qG1m8KfX1A460l2x++WtusOr7hRGd4IpyTuPu/eHmE6hqIJ3SvX4jxzQRHcffWnBIkieTZPr9QXPM0Lh6jn9EHvjNPKi3gtF7ZsS84iOMupKBbEESzOH0sUlx9e55+hkQyrgucEtQ4hh09GGBfzgKahuVkbpQqLlIwxFsk3wlTrwL4zQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aR7czHru4c3LjTEzuiK+4omMBrlfIsG0K7PTQ0RRnlI=; b=TkWf0Pd+DWFzBJ/vN86odYM8GZ6klNFeuHwYnrjJy9yhv/7TxXtAUSGqucBwhr1Uz33j9vr+u0Nm7DUagM2T/e2MSh+if/hqoISI3nWUp2RfWZFR+cgYMmEKLcBxuKsTwLRW3k5U6bZtpqMEVhMI5s/9j2e5FLispWb304ptzC+YAuL77jdh9YNeCdxCczYntfmirnAgD03uyk4OwRogDTsAAunf+4eap4npJcnn7lSqLzzA73SvsxrUiLYQ2/PrX0Uc/YOcmd2ZW4ue85ywKTnMWZM8jS+oic0I/ysWNtFvWakrgDG0lm2FMsP/0l56fAcR1osQPSQeYokfscbPbg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aR7czHru4c3LjTEzuiK+4omMBrlfIsG0K7PTQ0RRnlI=; b=kGH5hlUYKw0o7vSbvXwdxft5qZ8GrmTgtaPAfDb4NUvKWQKS+0hjyoowwXpx0By6RpvnuwLnBT2pzAlvgX66iRvixYyb4WZ1qgI3b0+92CDFe7UqU/Q1IhG0CGz/wiNy1DtsFdFliofH/DaOiZgDPPNHnjQ/PuFkUWogmDBYZc4= Authentication-Results: mellanox.com; dkim=none (message not signed) header.d=none;mellanox.com; dmarc=none action=none header.from=mellanox.com; Received: from HE1PR05MB3484.eurprd05.prod.outlook.com (2603:10a6:7:2f::12) by HE1PR05MB3259.eurprd05.prod.outlook.com (2603:10a6:7:2e::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3066.20; Fri, 5 Jun 2020 03:07:14 +0000 Received: from HE1PR05MB3484.eurprd05.prod.outlook.com ([fe80::3022:cbe2:4523:718a]) by HE1PR05MB3484.eurprd05.prod.outlook.com ([fe80::3022:cbe2:4523:718a%4]) with mapi id 15.20.3066.018; Fri, 5 Jun 2020 03:07:14 +0000 From: Suanming Mou To: Matan Azrad , Slava Ovsiienko , Ori Kam Cc: dev@dpdk.org, rasland@mellanox.com Message-ID: <3210893d-13c8-d3e6-9ada-0db2e2165307@mellanox.com> Date: Fri, 5 Jun 2020 11:06:45 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.8.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-ClientProxiedBy: SG2PR06CA0185.apcprd06.prod.outlook.com (2603:1096:4:1::17) To HE1PR05MB3484.eurprd05.prod.outlook.com (2603:10a6:7:2f::12) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [192.168.31.73] (218.74.52.164) by SG2PR06CA0185.apcprd06.prod.outlook.com (2603:1096:4:1::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3066.18 via Frontend Transport; Fri, 5 Jun 2020 03:07:12 +0000 X-Originating-IP: [218.74.52.164] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 9fb783a1-e2fa-42b6-d875-08d808fd8c04 X-MS-TrafficTypeDiagnostic: HE1PR05MB3259: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtFwd X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-Forefront-PRVS: 0425A67DEF X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xZlSTCXwiM033pyJMO7pTnRN9J3EB8RKxi/l2tYAV7HhPp0mnqgG8Bjhnv5ObzBefZYSGjHNDEz6f8Hd6wcrHWU5MsvEJGMYWKebB7/er4nHwR9dnwgIi71qU5qgTHYvSpj8bi9jmk0ufR2WUDAVO/ZBBmZkHfmIPbPxJ3ZsaPWbFC2U/AvfAb/EtOrObNDbLUGMAbz/TvoKnisabEy+IGGxp0PrB9gsSzXQ1zr2Die3BwsoPjm4gYp3ucxCbwhCSNBHln+3htocQ9Bgs62r94ZKtMvb2n30xG2dHA0wzk5bJN55VzCsXJdgP3McAb7dhOtiX8p4Cl2NDN0Ghi+SAff8VOs2a1hghH3nLtL4deEMgfyckfiL3ZO0x5fDiv2Y X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:HE1PR05MB3484.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(396003)(366004)(346002)(376002)(39860400002)(956004)(186003)(4326008)(6636002)(66946007)(16526019)(66476007)(66556008)(36756003)(6666004)(2616005)(8936002)(8676002)(16576012)(316002)(52116002)(26005)(478600001)(107886003)(6486002)(86362001)(110136005)(2906002)(5660300002)(31686004)(31696002)(83380400001)(43740500002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: oyPe9TPlYRW8IWWFqIHzwPAj4u5xWrz5mPV80amcmRagoqBVOqTIle6MbwxTOnSD3e7A4s+5c98YkA+sRtvgdYBu6WCDpgo7uoqTH21gkbN6AeowX2EOKM+p+QBa38I2TO2F8QAfqItr2ZnAbJB+iO4aKOt3mndylMzBAwkQXGSmNFPTGK7lXYN4xs4q/gM5KqNWPZjF2h77y0Jqo11kCfnFGgJL0uhIePGnOyEmc4jZMjrAxjd4IGJ8zgD+5JR0g/C9NayDZVSO3mpzWzl1DCHDxul/iM5+V6bGEHDINZalqyJMloBMP1YhLK8O5iaQjMOfcClx56io+bW0Yvvc2/rKUHLQgIOOa7xT3fltPEHU9Pl39UPfEvcGkiZk6589HMHL0vqhwNsXu7MqBS2BW+aqvWwNnn4Ceu8sXj/omcwtEnHVb/ekn4nXkwPiqq1rxAElQ72qC3dWgu5wwZodPqwIMK19ygOmdiqDQedzGEY= X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9fb783a1-e2fa-42b6-d875-08d808fd8c04 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jun 2020 03:07:14.2819 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: +OLc1qTB/yXZDYq0NPnY+fFT6HeYXh2/L6ofgPR8yVPfun0wy0UX3Pn78zHo4sIggJdQxMXJj+0VQHBDMwvpAw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR05MB3259 Subject: [dpdk-dev] [RFC] net/mlx5: add sys_mem_en devarg X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Currently, for MLX5 PMD, once millions of flows created, the memory consumption of the flows are also very huge. For the system with limited memory, it means the system need to reserve most of the memory as huge page memory to serve the flows in advance. And other normal applications will have no chance to use this reserved memory any more. While most of the time, the system will not have lots of flows, the reserved huge page memory becomes a bit waste of memory at most of the time. One solution maybe not to use the huge page memory for the DPDK application, but in this case it hurts the datapath performance. Since the flows memory is just for the control path, allocate the memory from system will only affects the flow insertion rate, so the other solution becomes provide a new option which allows flows allocated memory from the system memory. In this case, for the system with limited memory with this option, it gets a balance with the huge page memory consumption and datapath performance not be hurted. This feature will add a new devarg for MLX5 PMD which calls "sys_mem_en". With this devarg set to be true, the system memory allocate functions will be registered for flow objects memory allocate. The "sys_mem_en" devarg can be set to 0 or 1. 0 means the flow objects will allocate from rte memory. 1 means the flow objects will allocate from system memory. The sample configuration as below: ./testpmd -n 4  -w 0000:d8:00.0,sys_mem_en=1 -- --burst=64 --mbcache=512 -i  --nb-cores=7 For DPDK application without this devarg set, it is same with "sys_mem_en=0", the default memory allocator for flow objects will still be the default DPDK rte memory allocator. Thanks, SuanmingMou