From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-CO1-obe.outbound.protection.outlook.com (mail-co1nam03on0053.outbound.protection.outlook.com [104.47.40.53]) by dpdk.org (Postfix) with ESMTP id 7466D1BB06 for ; Sat, 23 Jun 2018 04:24:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amd-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QI5DX0aov2SjpGhgjgyhNBUYGjaDWHo9z7wtoFy8nzw=; b=LOyL7PBYd1H6gKdrC3AONUAJF6pfHrK/3bjxopBnFp3IMSjBWObW7E3uOPiYXj51Ttn8uJxvCNVbXKdk8HFKDm1iTLKsyCSF6+//92/gQRrIsLXX3PohHkOsGEdTG4Y2EKAEf80vJpzPhRO74y4rDGgKkFwiuK1H/iuP7SwmkRM= Received: from CY4PR12MB1509.namprd12.prod.outlook.com (10.172.68.22) by CY4PR12MB1896.namprd12.prod.outlook.com (10.175.82.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.884.20; Fri, 22 Jun 2018 16:37:18 +0000 Received: from CY4PR12MB1509.namprd12.prod.outlook.com ([fe80::61bb:2029:db88:f8a0]) by CY4PR12MB1509.namprd12.prod.outlook.com ([fe80::61bb:2029:db88:f8a0%6]) with mapi id 15.20.0863.021; Fri, 22 Jun 2018 16:37:18 +0000 From: "Kumar, Ravi1" To: "dev@dpdk.org" Thread-Topic: DPDK 18.05 only works with up to 4 NUMAs systems Thread-Index: AdQKRzR0A6b9yu3fQ5STQvinepV22A== Date: Fri, 22 Jun 2018 16:37:17 +0000 Message-ID: Accept-Language: en-IN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Ravi1.Kumar@amd.com; x-originating-ip: [122.167.114.246] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; CY4PR12MB1896; 7:dJDM1auCmKnu4bWdWZfsaT8HF1Gc0KvQJvbwslLKtOILP4NhUsp0EwtBmZaHJaK24X2RfLOD3k5EQl/l0LKa5pYBrNC/MZtLhHk6iIe5c9C0102TdcFPRA55dHAzwUKhLYYVG/8YCIOsZo+JECndFWjr5WK/nkVAjcnLSBSWD43A4u04wIN2+D1YbLolKuut/9bg9qK/AxOMaRlmu6zme0LPq2Xi9RjeVHrB2NO2vuUBxZNTEJWOoyYV3cQ5tTEd; 20:fETEPCAVd3evKEBEkp7EoRpbAYHIPx4SZjoi4UGnzwYlJmh5pJyHPD9koY1/RCm08voWNu98d28kMLUlw5nqla0qdyr0TwadmEe3Udryu9dGu6VERe8g4VX13Zp29zQkKxLtjXZfmrNk2E7WWPe3Q94zVRyhLTA3ihQBsh3a06zCr+PvogjhcG3iVrNcK/1AoiZssl2Un+GeRMUj4unVq3VCKOyslYfo1w2DSrQMcGYXGx3n08Dp11l/hI0Xhe5F x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: 08f81540-2453-466b-e5ed-08d5d85e6b3e x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(8989114)(4534165)(4627221)(201703031133081)(201702281549075)(8990104)(5600026)(711020)(48565401081)(2017052603328)(7153060)(7193020); SRVR:CY4PR12MB1896; x-ms-traffictypediagnostic: CY4PR12MB1896: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(28532068793085)(21748063052155); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(93006095)(93001095)(3231254)(944501410)(52105095)(10201501046)(3002001)(6055026)(149027)(150027)(6041310)(20161123558120)(20161123560045)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(6072148)(201708071742011)(7699016); SRVR:CY4PR12MB1896; BCL:0; PCL:0; RULEID:; SRVR:CY4PR12MB1896; x-forefront-prvs: 071156160B x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(346002)(366004)(39380400002)(396003)(39860400002)(376002)(199004)(189003)(501624003)(7736002)(68736007)(14454004)(9326002)(97736004)(7696005)(86362001)(2351001)(105586002)(478600001)(72206003)(5630700001)(5250100002)(25786009)(6916009)(106356001)(2900100001)(33656002)(5660300001)(66066001)(9686003)(6436002)(2906002)(316002)(6306002)(99286004)(2501003)(790700001)(54896002)(26005)(476003)(186003)(6116002)(8936002)(102836004)(1730700003)(3280700002)(6506007)(3846002)(5640700003)(74316002)(53936002)(8676002)(81166006)(3660700001)(486006)(81156014)(55016002); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR12MB1896; H:CY4PR12MB1509.namprd12.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: amd.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: M3hBwm+6Ut3iBtAvx0cDNQAgRDF7vet2bkD/eMMdhMHuL6dn/isuGOF2LqUzYRbi5sBjUYjUg6E05HZkFvEqbUYtMpDZERRtC/sQaD+xEGH+GbHP5fU+hIF4mRgHHkGyIvuAuIjEJYubJN5/7w661XSc8YV/XLLNMj6lRZFDJBfrY9plkQkZPVk8sUiEsWjOBvi2Yn7Fhw+pGad5tAwFTTRyQ9mSShyX2o29wNjtdFOfWQaieRyw4CndvGaUoQVifQyxF6yY0P8/Zm72A6+Euhk7X+McAyH8Fdeq8pRf/iltXw0HI+H2Rdh1LdQ31Tsmk0K/2PobJJ9hdc3SGoFwvA== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 08f81540-2453-466b-e5ed-08d5d85e6b3e X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jun 2018 16:37:17.9352 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1896 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] DPDK 18.05 only works with up to 4 NUMAs systems X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Jun 2018 02:24:34 -0000 Hi, As the memory subsystem in DPDK 18.05 is reworked, it has introduced a prob= lem for AMD EPYC 2P platforms. The issue is that DPDK 18.05 only works with up to 4 NUMAs. For AMD EPYC 2P= platforms, DPDK now only works with P0 (NUMA 0-3) and does not work with P= 1 (NUMA 4-7). The problem can be fixed by reducing some of the default settings of the me= mory subsystem. To solve this issue: - We can create our own config file for our integrated 10G NIC= , that is for amd_xgbe PMD. This will make amd_xgbe immune to this problem. - However, when any other NIC (Intel, Mellanox, Cavium or Broa= dcom etc.) is plugged into NUMA 4-7, the problem will still be exposed. - If we only fix it in "config/common_base", it will cover all= cases. Our current workaround is: Edit config file "./config/common_base" and change the following line CONFIG_RTE_MAX_MEM_MB_PER_TYPE=3D131072 TO CONFIG_RTE_MAX_MEM_MB_PER_TYPE=3D65536 Any better solution for this issue is welcome. We would appreciate if this issue can be fixed in the next release (18.08) = so the STOCK version of DPDK works on AMD EPYC 2P platforms. Regards, Ravi