From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6C8ECA0032; Wed, 14 Sep 2022 09:31:03 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1796740151; Wed, 14 Sep 2022 09:31:03 +0200 (CEST) Received: from rcdn-iport-9.cisco.com (rcdn-iport-9.cisco.com [173.37.86.80]) by mails.dpdk.org (Postfix) with ESMTP id A522240141 for ; Wed, 14 Sep 2022 09:31:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=34036; q=dns/txt; s=iport; t=1663140661; x=1664350261; h=from:to:subject:date:message-id:mime-version; bh=1zM9nNIJB8WUKFcCNyUvN7cmd37rkuE3DJiwThe7BD8=; b=mt06IfNDw/ID6UCsJac2d/SV3KTUww72CWABmEv3f0lyo+Bt5IwAIfj7 4XWVCoMx/gu4GCM54v/5TGWNMPrK7LSHkF/pYnn760ynmPUzwkl7OGqqv KIQlKZFV3kdcVK89po9i0akD4OJzUi4VlEyKiPlZrgZ9fdloG+mm6usAZ 0=; IronPort-PHdr: =?us-ascii?q?A9a23=3Ap1s+vxEofVR9fIXV4cF0TZ1GfiYY04WdBeZdw?= =?us-ascii?q?pYkircbdKOl8tyiOUHE/vxigRfPWpmT8PNLjefa8sWCEWwN6JqMqjYOJZpLU?= =?us-ascii?q?RJWhcAfhQd1BsmDBAXyJ+LraCpvGsNEWRdl8ni3PFITFtz5YgjZo2a56ngZH?= =?us-ascii?q?RCsXTc=3D?= IronPort-Data: =?us-ascii?q?A9a23=3AASb8paocGDKSVViVuWE2oxbY4kZeBmItZxIvg?= =?us-ascii?q?KrLsJaIsI4StFCztgarIBnVMq6DZWOgco90bovj8EoO7cKAyIJkQVFv/yk3Q?= =?us-ascii?q?XgQ9ePIVI+TRqvS04x+DSFioHqKZKzyU/GYRCwPZiKa907F3oTJ9yEmj/nVH?= =?us-ascii?q?+SkUYYoBwgoLeNaYHZ54f5cs7ZRbr5A2bBVMivV0T/Ai5S31GyNg1aYBlkpB?= =?us-ascii?q?5er83uDihhdVAQw5TTSbdgT1LPXeuJ84Jg3fcldJFOgKmVY83LTegrN8F251?= =?us-ascii?q?juxExYFA9ehlPPwdVcHB+KUNgmVgX0QUK+n6vRAjnVtieBgarxFMgEO0Grhc?= =?us-ascii?q?9NZkL2hsbStRgAlN7PFgswWUgJTFGd1OqguFLrvcCfu4ZXKlBGfG5fr67A0Z?= =?us-ascii?q?K0sBqUU9/hfDXlC9rofMj9lRhmFjv6xxKP9QPR2j8ckMuHqOp8SvjdryjSxM?= =?us-ascii?q?BqMafgvWI3D4dtemTw3nM0LQbDVZtESbnxkaxGoXvGGAX9PYLpWoQtiriCXn?= =?us-ascii?q?+VklW+o?= IronPort-HdrOrdr: =?us-ascii?q?A9a23=3Ary9x6aEsWchS2QyZpLqFY5HXdLJyesId70?= =?us-ascii?q?hD6qkvc3Jom52j+PxGws526fatskdrZJhSo6H7BEDgewKVyXcR2+gs1NiZLX?= =?us-ascii?q?DbUQeTXeNfBOjZsnbd8k/Fh5ZgPM5bGsAUYrCRfDtHZK3BkW2F+qMbsb+6Gd?= =?us-ascii?q?eT9IDjJhlWPGRXQpAlyz08JheQE0VwSgUDL4E+DoCg6s1OoCflUWgLb+ygb0?= =?us-ascii?q?N1FdTrlpnurtbLcBQGDxko5E2lljWz8oP3FBCew1M3Ty5P+7E/6mLI+jaJq5?= =?us-ascii?q?lL8svLhiM05VWjoai+q+GRi+erw/b8yvT9Hw+cxTpAor4RGIFq8gpF4t1Ho2?= =?us-ascii?q?xa7eUk6y1QQ/ibrUmhO11cZXDWqk7dOPFE0Q6n9bbQuwqdneXpAD09EMZPno?= =?us-ascii?q?Rfb1/Q7Fchpsh11OZR03uerIc/N2K2oM3R3am8a/hRrDvBnVMy1eoIy3BPW4?= =?us-ascii?q?oXb7Fc6YQZ4UNOCZ8FWCb38pouHuViBNzVoK8+SyLSU1nJ+m10hNC8VHU6GR?= =?us-ascii?q?mLBkAEp8yOyjBT2HR01VERysATlmoJsJg9V55H7eLZNbkArsA5cuYGKaZmQO?= =?us-ascii?q?sRS8q+DWLABRrKLWKJOFziULoKPnrcwqSHkondJNvaC6Dg4KFC6KgpCmkoy1?= =?us-ascii?q?LaU3ieePGz4A=3D=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0BdBgDea4Ji/4kNJK1aHgE8DAILFYF?= =?us-ascii?q?PgSExUgd1Alg5Q4gaA4UxhQldgzuaJYEsgSUDVAsBAQENAQFCBAEBhQIChT4?= =?us-ascii?q?CJTQJDgECBAEBARIBAQUBAQECAQcEgQkThWgNhlUGFRkBATgRAUABPycEARo?= =?us-ascii?q?aglyCDFcDMQGfdQGBPgKKH3iBATKBAYIIAQEGBASFDRiCOAmBPIMUhCeHTBy?= =?us-ascii?q?BSUSBFUOCMIU7hAuCLpVhBzoDVIEFEoEhcQEIBgYHCgUyBgIMGBQEAhMSTQY?= =?us-ascii?q?eAhMMCgYWDkISGQwPAxIDEQEHAgsSCBUsCAMCAwgDAgMuAgMYCQcKAx0IChw?= =?us-ascii?q?SEBQCBBMfCwgDGh8tCQIEDgNDCAsKAxEEAxMYCxYIEAQGAwkvDSgLAwUPDwE?= =?us-ascii?q?GAwYCBQUBAyADFAMFJwcDIQcLJg0NBCMdAwMFJgMCAhsHAgIDAgYXBgICGVg?= =?us-ascii?q?KKA0IBAgEGAQeJRMFAgcxBQQvAh4EBQYRCQIWAgYEBQIEBBYCAhIIAggnGwc?= =?us-ascii?q?WNhkBBV0GCwkjFgYcAQ8RBQYWAyZSBQQflw2CFIFlOqBTQ58QgTAKg0ygJhW?= =?us-ascii?q?DdYw+mCSWZiCmXgIEAgQFAg4BAQaBYTyBWXAVO4JoURkPjiwWg1CKXnU7AgY?= =?us-ascii?q?LAQEDCZEaAQE?= X-IronPort-AV: E=Sophos;i="5.91,230,1647302400"; d="scan'208,217";a="979811341" Received: from alln-core-4.cisco.com ([173.36.13.137]) by rcdn-iport-9.cisco.com with ESMTP/TLS/DHE-RSA-SEED-SHA; 14 Sep 2022 07:30:59 +0000 Received: from mail.cisco.com (xfe-rtp-003.cisco.com [64.101.210.233]) by alln-core-4.cisco.com (8.15.2/8.15.2) with ESMTPS id 28E7UsEx003317 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=OK); Wed, 14 Sep 2022 07:30:59 GMT Received: from xfe-rtp-002.cisco.com (64.101.210.232) by xfe-rtp-003.cisco.com (64.101.210.233) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.9; Wed, 14 Sep 2022 03:30:55 -0400 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (64.101.32.56) by xfe-rtp-002.cisco.com (64.101.210.232) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.9 via Frontend Transport; Wed, 14 Sep 2022 03:30:55 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EMcQs5JSsl7+kSpINx2LZCBSb4/HZrVyEAUlvieI3jYRulg+hz4yzaU5p5QK9m2qlxJtbyRZYp3K+RfSkHf7y9qf/sCjF4Vu6xzKH0rUnLdWeuR7PQRPz9v4UbkpkCOX01C/HPB/nA49/nuwRSLSuR6cM1gJKWDE0rZKY1i5rg+fBp3woCFKlWmSzoN6qIdEEUUlzhm9aKCeBv+4RfuekV0twhFUGOhi95HYlmsTfP7SyyNPVvwLaKzLhjJ1m+VC3zW0Q1oSyinRV+AfU2CD9Ryu0CBEEZ7r1qIrJwIdt8iUkwE8CNiSNx+6jsbhLTmiEQDd31rGe774BmauKoR8fw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=M9YrfxqgX6GRD3r8jBN56WINWuIoqptwv4C9CCgOHR0=; b=jl2Au+3auWycFOf8CtaCnwJoHLAJRlZpDCDHAgtL9ohJPUmoSuaD1sWBi8xjqoXBHZ9UGfvsq3ciCPg8E+d222eBB4LiiVFD8YhYgQ6m66X6ttrIoFUPVejKFP8EjTV4RA0ZP141ggz1fOHOLiJnNlTml2+YPZb82eHI8ZlP8sciwh/pnX3ZOm8Y+p+0FHB0R5lc8CR9JomXLG1LB2QyZ7sSGh6wPlIUW9c9H5d4cpBbXiDtK+JXnAGUfkj1dpe2vBf1gIL0WvtwbWiHqT5CAl5Vu6CxmguXz4MEyDhfxROXUDIbIEeiLL6iYG5mtay9Z3yPTxQYElS0CEbiGosMdQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cisco.com; dmarc=pass action=none header.from=cisco.com; dkim=pass header.d=cisco.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.onmicrosoft.com; s=selector2-cisco-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=M9YrfxqgX6GRD3r8jBN56WINWuIoqptwv4C9CCgOHR0=; b=XwyVcHUJqXrjX1lmXAX1ZzzN1wkvf4CMhufe+XoVXlXeH4xeCX2mZMrCXGetxmTRnm4PVPvL1AirWqZdoQ4GVg7oz9rHRgJ4aI1ydMenlV6DLC2/Vc6i5xyhzZAMGaxvaynDOLhAZoMZovTH/n8Tg6j+fRCAzZa4OprFm6BKQlw= Received: from SJ0PR11MB4846.namprd11.prod.outlook.com (2603:10b6:a03:2d8::24) by PH8PR11MB6778.namprd11.prod.outlook.com (2603:10b6:510:1c9::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.22; Wed, 14 Sep 2022 07:30:54 +0000 Received: from SJ0PR11MB4846.namprd11.prod.outlook.com ([fe80::85f:e815:92f9:7073]) by SJ0PR11MB4846.namprd11.prod.outlook.com ([fe80::85f:e815:92f9:7073%8]) with mapi id 15.20.5612.022; Wed, 14 Sep 2022 07:30:54 +0000 From: "Umakiran Godavarthi (ugodavar)" To: "anatoly.burakov@intel.com" , "dev@dpdk.org" Subject: DPDK 19.11.5 Legacy Memory Design Query Thread-Topic: DPDK 19.11.5 Legacy Memory Design Query Thread-Index: AQHYyAmT17eWxN4VFkO+VD/TnQXrtA== Date: Wed, 14 Sep 2022 07:30:54 +0000 Message-ID: Accept-Language: en-US Content-Language: en-IN X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=cisco.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SJ0PR11MB4846:EE_|PH8PR11MB6778:EE_ x-ms-office365-filtering-correlation-id: 1cb82ac2-ea14-47d8-bf8c-08da96230ee2 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: F9+sPg/y2+gw22oqXE9rN5As9TdX4KzkB9uqO4ZeUQopQJw3VUjKO2Mvu28Y/Q+EtP/ARReFxxUAvrmjebtftGVlv8IVB1TVRM5a/WwcnG/F0D1DfLX1xgpO6BOkhiRgCB3NXXVqq+4h8wWPHajbJ1uZ1yyMvQQfkscbNa1tAF20kaCCmpxx0z09Jr8n5KTXwPRGZR8DP4Y0moOWzznSUG7UZoUJGlFko/Dx5FBMkzxbWvNCQHoCrHX2bfzUhR8GbeVr/lPsEhGLiWgCi7X61xoSq+pl8vhC4Cr/lMEbn+qv1LEe7S8e/dQEu7moZysYR/ky7b7ZL3cCJf8FzxOEibTkxzcfnUHXZy9QvA0Eo4ONpDIj5Oiei8UibwIu6Cgbame66rTc1xnXnrTPrZrxs2HcCEDx7wrfwcEvll3smbJsEXHQCFR3WODNWV+/OfgRFUhRDSfoYWGDEqY4Pj3OH8a35EcI9hhIW133WqAnxFo5TW/nbdKnaqnmAzb3/Pio/c27O1gsvByrj0+nCx9HbL3rFcb3Wb/z9Q3bsHckkYvKjBiktF8t8D1GN11C8Fzl/I/VrtGuAumb+r1ilaHID18Scygfe0VizLcCpAbMrfAioJGz6ChNMk9/21zmw+N4wBozEC1iDNrkAj9LYPtOoOcIsSzZfY6OLH7iTM5yOEEqduS1qrbxs5gN/0OpUG3tD3bbpH1CbcwAJY2wT1ir+PO1pFLpIzXr/i0gUv1DLAKLtW81GU3NXisfuf1hPY0EQbwR88R3CoLFdQ0f37yFjw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR11MB4846.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(39860400002)(346002)(136003)(376002)(366004)(396003)(451199015)(122000001)(52536014)(66946007)(33656002)(8676002)(66446008)(186003)(66476007)(55236004)(64756008)(7696005)(5660300002)(478600001)(91956017)(38070700005)(55016003)(66556008)(38100700002)(41300700001)(86362001)(8936002)(26005)(110136005)(2906002)(6506007)(316002)(71200400001)(76116006)(9686003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?Windows-1252?Q?gjZm4V8TdHgY/+siyxAaxtcaaB5LWrnLouVRsZtf+iZ4o6rO19iHxrrQ?= =?Windows-1252?Q?Zr442Sa9vqxkSB6nQscguWA9QNdvkzDM3srADVijO3EZ2dSICL2Qm+X+?= =?Windows-1252?Q?Q+kyyI7Ut+yxrFpYccP94w7npLpsNx12RqRfwK1GAZ7+87trYOGvkJGy?= =?Windows-1252?Q?U9LoRnuy3J/A8vk3rKmrsKJ5Cm/POMlCNUfN2ZifCqVrhnAxE6XrQnws?= =?Windows-1252?Q?MGlDQ+F3/JsoyzV5kYA3df5xFPgk//cqyIMF7WTgVY4nUBwvYBH0lAtK?= =?Windows-1252?Q?fVz3zryy6RMZ30ztTOzy8vTpeNuknMV9WALMoX9JaoBDUeIr19oE3Bpu?= =?Windows-1252?Q?6+Umsq14kwdVNrnECIrRs4Fh/KQryLw6G/C114NUw17po/QAwa2Aq7uR?= =?Windows-1252?Q?REZbyrodgpXIWlgLMvQEPPh9vUHKi7NtU9isn/WdJI8Y0zyDSbwA04lf?= =?Windows-1252?Q?nzPGkwoWGxgK1m6sk239GcFTCeWiMRirvEpJqZVu6IiZzo2voNs0668T?= =?Windows-1252?Q?P2zfWT066zCp7lzMCFTEdM3etxZ3uqGH29IRzQcnNz9uqNOmrwUghpFG?= =?Windows-1252?Q?1bUZeUAuuBvPI8RiPR2+a2klgsmf7emhmt8KlgR3RRH3yfR3vR8a6ZlV?= =?Windows-1252?Q?bkYfSPiLJm1JRT5AezzgLsWvgRr+SP99nwFMEqSF7e+jEa4jGVdDFGQG?= =?Windows-1252?Q?EdYOotM7XlIeYs7e7BS+f218DvUBRq+7EVDd0q3oYpTpfiw6kTIhSPzJ?= =?Windows-1252?Q?/HWqc/GwoOh4TD/q8+Ne+BMtivpnas5qMnefNO911DYGET36OSRODy9t?= =?Windows-1252?Q?DQuwLMC97fDOvyqollN4L40UIRxXT9PoepQlgBlCW0BSZk5ov5Vl4GxM?= =?Windows-1252?Q?XkZIGFh2T2LFXyFgcbRF1PYjtsOpSohDKRtpU5MCBvMT6V9qhI9uY6PY?= =?Windows-1252?Q?ciqgPyZz/leAHh/i/6XhpGWXez5FuP/b1AwULAGBu1UR2IJqfeSLy7SF?= =?Windows-1252?Q?Z+QTt8Q5pyBLyENlseq9KR83bFsaln20RFrq9cQIIYFyFb+VAwCU2zQw?= =?Windows-1252?Q?7x4/cYlEfqLpYdn0LVhemRaqH2lMQniu1BAUqXTSqgzyWwLyYfWTb/Ql?= =?Windows-1252?Q?6PGFqnI3ITYuHigKgfPISXkWKrwOtUFamwT5/JUv/IWGKgnN50b9ujqk?= =?Windows-1252?Q?mJWqwTvoZsexTwMYaHtvaCi12iRAB5wWWOLqvkkZ4EgcUdesWJSTG970?= =?Windows-1252?Q?18si336hOWdtb77n0eJr2iRzdGRtEmh12QH6bKn6ct5AgNjTvaZ04vFQ?= =?Windows-1252?Q?SsLvTyFtGyCUtMP1u3izATMvonQsn398pN8uf3YF4w+AvH2yKoDTeqOT?= =?Windows-1252?Q?YH32YHSIocmSwiGNa7dw3s+ETjhtbqheHKWGaPwHyv/g8h30w5o9isSr?= =?Windows-1252?Q?6WO8rJRPC6YltUPNZ7GXofQmmOkzoyevKrZNr5pgHudFko4tGD7A+nSP?= =?Windows-1252?Q?7TwP0oeS1DcuvgmEbIJ3qYPibdM9L13k3wFde5SdRfC4eXloDzZBoMdU?= =?Windows-1252?Q?TG/pQPPH19Hm/KHuL5fZFv4quIX/0wF1VItADx52EASyKzXtnsH+oHmS?= =?Windows-1252?Q?oo5UaL7ZcWevSfQQyrJYH+m2r+b9sQOfkOTyLSlrIXz6L/QcZ6DadpRD?= =?Windows-1252?Q?upvfqHCPyT9fubjIWq76MAJUTCa9SENCKlqbW9zc/ViM1wRkXoOzbw?= =?Windows-1252?Q?=3D=3D?= Content-Type: multipart/alternative; boundary="_000_SJ0PR11MB484677B272ABA8D4C336925CDD469SJ0PR11MB4846namp_" MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SJ0PR11MB4846.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1cb82ac2-ea14-47d8-bf8c-08da96230ee2 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Sep 2022 07:30:54.3952 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 5ae1af62-9505-4097-a69a-c1553ef7840e X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: rRbHpWLh7cHkRGPevpx365LOC1fe4FIem4lb7RyQEAYX+JoitzFWEKRrj5/z0Xcsn60x0i9Uv2i6bBCX9sUC8g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR11MB6778 X-OriginatorOrg: cisco.com X-Outbound-SMTP-Client: 64.101.210.233, xfe-rtp-003.cisco.com X-Outbound-Node: alln-core-4.cisco.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --_000_SJ0PR11MB484677B272ABA8D4C336925CDD469SJ0PR11MB4846namp_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Hi Anatoly/DPDK-Developers I am working on DPDK 19.11.5 Legacy Memory design and have a query about ho= w to boot up in Legacy memory mode. 1. Linux kernel boots up with huge pages (=91N=92) and free huge pages (=91N= =92) initially 1. We calculate huge pages we need for data path (Driver need buffers fo= r all queues) by sorting the memory fragments 2MB huge pages fragments and = find we need =91X=92 pages , so we go to kernel and set SYSFS attribute nr_= hugepages (/proc/sys/vm/nr_hugepages) to X pages. We also store the sorte= d physical memory fragments as POOL_0, POOL_1, POOL_2=85.etc. (Just for poo= l count purpose enough for all ports and queues to initialize) For example, if host has memory pattern huge pages like this for total 500 = we get in step 1 kernel reservation. 250, 90, 80, 70 , 10 -> Sum is 500 (N pages) We need only 350 pages (350 based on no of ports, queues dpdk application n= eeds) So we need 250, 90, 10. So total 3 pools POOL_0 -> 250 pages, POOL_1 -> 90, POOL_2 -> 10 pages 1. We boot up DPDK by RTE_EAL_INIT 1. Then we go to DPDK Memory segment list walkthrough and for each FBARR= AY , we find the used pages by DPDK and unmap the remaining pages by below = code (Idea is to free the huge pages taken by DPDK process virtual memory) = -> Free_HP will be 0 then, as X pages are used by DPDK and all unnecessary = pages are freed in this step) Sample Code of 4 : rte_memseg_list_walk_thread_unsafe(dpdk_find_and_free_unused,= NULL); ->DPDK_FIND_AND_FREE_UNUSED is called for each Memory segment list = (FBARRAY pointer is derived from MSL like below) dpdk_find_and_free_unused(const struct rte_memseg_list *msl, void *arg UNUSED) { Int ms_idx; arr =3D (struct rte_fbarray *) &msl->memseg_arr; /* * use size of 2 instead of 1 to find the next free= slot but * not hole. */ ms_idx =3D rte_fbarray_find_next_n_free(arr, 0, 2); if (ms_idx >=3D 0) { addr =3D RTE_PTR_ADD(msl->base_va, ms_idx * msl->p= age_sz); munmap(addr, RTE_PTR_DIFF(RTE_PTR_ADD(msl->base= _va, msl->len), addr)); } } 1. With NR_PAGES As =91X=92 and FREE_PAGES as 0, we create MBUF pools us= ing RTE API and we face crash (We take care =91X=94 pages has multi pools b= ased on memory fragmentation given to the primary process, so pool by pool = we have confidence that DPDK should find physical memory segment contiguous= and allocate successfully) struct rte_mempool *pool; pool =3D rte_pktmbuf_pool_create(name, num_mbufs, RTE_MIN(num_mbufs/4, MBUF_CACHE_SIZE), MBUF_PRIV_SIZE, frame_len + RTE_PKTMBUF_HEADROOM, rte_socket_id() 1. Sometimes randomly we face a crash during pool create in Step 5 for e= ach POOL stored in Step 2 process for all ports and queues initialize later= on DPDK EAL core comes with BT like this #6 sigcrash (signo=3D11, info=3D0x7fff1c1867f0, ctx=3D0x7= fff1c1866c0) #7 #8 malloc_elem_can_hold () from ./usr/lib64/dpdk-19/librte_eal.so.20.0 #9 find_suitable_element () from ./usr/lib64/dpdk-19/librte_eal.so.20.= 0 #10 malloc_heap_alloc () from ./usr/lib64/dpdk-19/librte_eal.so.20.0 #11 rte_malloc_socket () from ./usr/lib64/dpdk-19/librte_eal.so.20.0 #12 rte_mempool_create_empty () from ./usr/lib64/dpdk-19/librte_mempoo= l.so.20.0 #13 rte_pktmbuf_pool_create_by_ops () from ./usr/lib64/dpdk-19/librte_= mbuf.so.20.0 #14 rte_pktmbuf_pool_create () from ./usr/lib64/dpdk-19/librte_mbuf.so= .20.0 #15 dpdk_create_mbuf_pool (mem_chunk_tbl=3D0x555d556de8e0 , num_mbufs= =3D46080, frame_len=3D2048, name=3D0x7fff1c1873c0 "DPDK_POOL_0") We see find suitable element does not able to find a suitab= le element in DPDK memory segment lists it searches for HEAP ALLOC and retu= rns NULL and NULL dereference crashes boot up process Please let me know any comments on boot up process for 1-6 and any reason b= ehind the crash ? We are suspecting Step 4 where FBARRAY unused pages freeing at last should = free the least contiguous memory segments right ? (munmap after finding 2 f= ree pages , entire length we unmap in step 4 to free virtual memory) Please let me know thoughts on FBARRAY design, is it expected to map the mo= st contiguous=85..least contiguous in a virtual address space right ? So our most contiguous segments in Step 2 is safe even after Step 3, Step 4= we believe. Please correct my understanding if anything wrong. Thanks Umakiran --_000_SJ0PR11MB484677B272ABA8D4C336925CDD469SJ0PR11MB4846namp_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable

Hi Anatoly/DPDK-Developers

 

I am working on DPDK 19.11.5 Legacy Memory design and h= ave a query about how to boot up in Legacy memory mode.

 


  1. Linux kernel boots up with huge pages (=91N=92) and free huge pages (=91N= =92) initially 

 

  1. We ca= lculate huge pages we need for data path (Driver need buffers for all queue= s) by sorting the memory fragments 2MB huge pages fragments and find we need = =91X=92 pages , so we go to kernel and set SYSFS attribute nr_hugepages (/p= roc/sys/vm/nr_hugepages) to X pages.   We also store the s= orted physical memory fragments as POOL_0, POOL_1, POOL_2=85.etc. (Just for pool count purpose enough for all ports and queues to initialize= )

 <= o:p>

For example= , if host has memory pattern huge pages like this for total 500 we get in s= tep 1 kernel reservation.

 <= o:p>

250, 90, 80, 70 , 10 -> Sum is 500 (N pages)

 

We need only 350 pages (350 based on no of ports, queues dpdk a= pplication needs)

 

So we need 250, 90, 10.

 

So total 3 pools POOL_0 -> 250 pages, POOL_1 -> 90, POOL_2 -&g= t; 10 pages

 

  1. We bo= ot up DPDK by RTE_EAL_INIT 

 

  1. Then = we go to DPDK Memory segment list walkthrough and for each FBARRAY , we fin= d the used pages by DPDK and unmap the remaining pages by below code (Idea is to free the huge pages taken by DPDK process = virtual memory) -> Free_HP will b= e 0 then, as X pages are used by DPDK and all unnecessary pages are freed i= n this step)

Sample Code= of 4 :

 <= o:p>

      &nbs= p;       rte_memseg_list_walk_thread= _unsafe(dpdk_find_and_free_unused, NULL); ->DPDK_FIND_AND_FREE_UNUSED is called for each Memory segment li= st (FBARRAY pointer is derived from MSL like below)

 

         &= nbsp;    dpdk_find_and_free_unused(const struct rte_memseg_l= ist *msl,

               =                     &nbs= p;     void *arg UNUSED)

         &= nbsp;     {

         &= nbsp;            Int= ms_idx;

         &= nbsp;             arr =3D= (struct rte_fbarray *) &msl->memseg_arr;

 

         &= nbsp;           &nbs= p;  /*

           = ;            &n= bsp; * use size of 2 instead of 1 to find the next free slot but =

           = ;            &n= bsp;* not hole.

          =              &n= bsp;*/

           &nbs= p;         ms_idx =3D rte_fbar= ray_find_next_n_free(arr, 0, 2); 

         &= nbsp;      

         &= nbsp;           if (= ms_idx >=3D 0) {

          =             &nb= sp;  addr =3D RTE_PTR_ADD(msl->base_va, ms_idx * msl->page_= sz);

             &n= bsp;            = ;  munmap(addr, RTE_PTR_DIFF(RTE_PTR_ADD(msl->base_va, msl->= ;len), addr));

         &= nbsp;            }

         &= nbsp;     }

 

  1. With NR_PAGES As =91= X=92 and FREE_PAGES as 0, we create MBUF pools using RTE API and we face cr= ash (We take care =91X=94 pages has multi pools based on memory fragmentation given to the primary process, so pool by poo= l we have confidence that DPDK should find physical memory segment contiguo= us and allocate successfully)

 

         &= nbsp;          struct rte_mempool *= pool;

 

          =          pool =3D rte_pktmbuf_pool_= create(name, num_mbufs,

               =                     RTE_M= IN(num_mbufs/4, MBUF_CACHE_SIZE),

               =                     MBUF_= PRIV_SIZE,

               =                     frame= _len + RTE_PKTMBUF_HEADROOM,

               =                     rte_s= ocket_id()

 

         <= /span>

  1. Sometimes randomly w= e face a crash during pool create in Step 5 for each POOL stored in Step 2 = process for all ports and queues initialize later on

     

         &= nbsp;     DPDK EAL core comes with BT like this

 

 &n= bsp;            = ;   #6  sigcrash (signo=3D11, info=3D0x7fff1c1867f0, ctx=3D0x7fff1c1866c0)

  &= nbsp; #7 

  &= nbsp; #8  malloc_elem_can_= hold () from ./usr/lib64/dpdk-19/librte_eal.so.20.0

    #9  find_suitable_element () from ./= usr/lib64/dpdk-19/librte_eal.so.20.0

    #10  malloc_heap_alloc () from ./usr= /lib64/dpdk-19/librte_eal.so.20.0

  &= nbsp; #11  rte_malloc_socket () from ./usr/lib64/dpdk-19/lib= rte_eal.so.20.0

  &= nbsp; #12  rte_mempool_create_empty () from ./usr/lib64/dpdk= -19/librte_mempool.so.20.0

  &= nbsp; #13  rte_pktmbuf_pool_create_by_ops () from ./usr/lib6= 4/dpdk-19/librte_mbuf.so.20.0

  &= nbsp; #14  rte_pktmbuf_pool_create () from ./usr/lib64/dpdk-= 19/librte_mbuf.so.20.0

  &= nbsp; #15  dpdk_create_mbuf_pool (mem_chunk_tbl=3D0x555d556d= e8e0 , num_mbufs=3D46080, frame_len=3D2048, name=3D0x7fff1c1873c0 "DPD= K_POOL_0")

 

         &= nbsp;      We see find suitable element does not a= ble to find a suitable element in DPDK memory segment lists it searches for= HEAP ALLOC and returns NULL and NULL dereference crashes  boot up process

 

Please let me know any comments on bo= ot up process for 1-6 and any reason behind the crash ?

 

We are suspecting Step 4 where FBARRAY unused pages fre= eing at last should free the least contiguous memory segments right ? (munm= ap after finding 2 free pages , entire length we unmap in step 4 to free virtual memory)

 

Please let me know thoughts on FBARRAY design, is it expected to map the most contiguous=85= ..least contiguous in a virtual address space right ?

 

So our most contiguous segments in Step 2 is safe even = after Step 3, Step 4 we believe. Please correct my understanding if anythin= g wrong.

 

 

Thanks

Umakiran

 

--_000_SJ0PR11MB484677B272ABA8D4C336925CDD469SJ0PR11MB4846namp_--