From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E8C5B41D56; Fri, 24 Feb 2023 04:02:54 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7BA8740693; Fri, 24 Feb 2023 04:02:54 +0100 (CET) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 07D8E40689 for ; Fri, 24 Feb 2023 04:02:51 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1677207772; x=1708743772; h=from:to:subject:date:message-id:mime-version; bh=RScfShyigH7bBAOyKAx3lgcHJ0pTzdJJBgu/6lDuy6k=; b=E4y+3l1xKKeVQSw/8Rpzm8Pny2GnasWJEhr7Q3R8iwU51hKj4LQ3viFC zNbZeA1BMMw05Qq3oRTwDxwjW0Wgqd1fV+8XQ89Hwoqv/pLhf6Xl5HEYZ bHECRNSGTPNr7FWsdooMrened3Pen2cnsoEvVavcb0l9Ffr3NuiDjU0fB RU0uRWhuCQZk2DGVFzqLqcvrbRmr+GcyXlNZEEnObpGKrQnB0SPJHyWvN k7hxP5QsDv9ctjNVSmmCZE67elY7Fa30QQKvdzm4ttsYsD78z+V7JdQnc Hh8K9f6YihQ/plwMsvyUMAjxFIv380CJFTQpppFdt6OTYBE654agnqD4U g==; X-IronPort-AV: E=McAfee;i="6500,9779,10630"; a="313020831" X-IronPort-AV: E=Sophos;i="5.97,322,1669104000"; d="scan'208,217";a="313020831" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Feb 2023 19:02:50 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10630"; a="1001671844" X-IronPort-AV: E=Sophos;i="5.97,322,1669104000"; d="scan'208,217";a="1001671844" Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83]) by fmsmga005.fm.intel.com with ESMTP; 23 Feb 2023 19:02:50 -0800 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16; Thu, 23 Feb 2023 19:02:50 -0800 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16; Thu, 23 Feb 2023 19:02:50 -0800 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.16 via Frontend Transport; Thu, 23 Feb 2023 19:02:50 -0800 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.176) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.16; Thu, 23 Feb 2023 19:02:49 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bANozo4f3CAl3p8UVY788YO3BjHMXEddziCCqCuqfkRO+byHPUer6imJW8hhaPlFGw0TJlh4CioI+NNgDsg6JAM1ikGLc89xA4tDyAFQ5DAZo79Oe91gUTw2STFtCF3SC/ceQdfhLA/PCmkpPEvpDp1mgn1gtcToAAKU3uIKCEixwo3/K0/0x5sD1ZAFqHtHOKoYAFayqy/zf3PzNvlgGfv3qsWvou5+1x8o8j3eI9ABIauQb+o2b538PzBAPtbC5HJJNNtIIDrEkegAKelctWqH5M8/JX3aDD/nh9L1LBJBYhvW3OaxmzVT8lgSGrffy51KV0SSPPpnfSYgAugI1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/55vcWWd01FupLG9xnXK8UGOApR39rart1sl37M+oXA=; b=fZW/I46uyGKVgLzbqE+xe8Cg9UzuRg7xtljh4WN7kBJ9YVB7/wSbdN34Hdis4Jh+eBkaHfSMtqgQ6daG5nzgBymdWeWhboUzCkM4vDJTNhXIhCL5Z3rVfiPw1YGy4fwqq59I9/AdxaJbpDJh3PQRztDi9yJ0mN0LUcVr/5AZZ6yswA5enTZwZ9J30F8l7mnaAXbU38RaaJ934GbOmWSTlVmetXq2z+O8EUZsUqGaZca8/75s5Ut7fnjmdFHU4jxwglygJElJp1jUtShgphHMkJqaw+6DpVF3QAwv2ry/VL5Yl0liFXIOjaxW5pVHTnrbOCIsquAq9fGQ57ZbPlgLig== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from BYAPR11MB3303.namprd11.prod.outlook.com (2603:10b6:a03:18::15) by CY5PR11MB6343.namprd11.prod.outlook.com (2603:10b6:930:3c::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6134.24; Fri, 24 Feb 2023 03:02:48 +0000 Received: from BYAPR11MB3303.namprd11.prod.outlook.com ([fe80::f1f2:fa10:9688:bf89]) by BYAPR11MB3303.namprd11.prod.outlook.com ([fe80::f1f2:fa10:9688:bf89%5]) with mapi id 15.20.6111.021; Fri, 24 Feb 2023 03:02:48 +0000 From: "Harris, James R" To: "dev@dpdk.org" , "mb@smartsharesystems.com" Subject: Bug in rte_mempool_do_generic_get? Thread-Topic: Bug in rte_mempool_do_generic_get? Thread-Index: AQHZR/oOTrogQubqxEK8xQ6tUeYFvQ== Date: Fri, 24 Feb 2023 03:02:47 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: BYAPR11MB3303:EE_|CY5PR11MB6343:EE_ x-ms-office365-filtering-correlation-id: ee0b5e3f-7312-427c-5a4f-08db16139bcc x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: jfdgO8AS6yy39k1+Epz0B6S/kD/HCPAkhMjP4Q5iTKLtlTJiS3qZlI1vNX8ZJ536mW/RDYltCvKCcjXansb/iupg8PPOW0ANazL084vYBvFUUrALPZejGjn1x2fYwgCDJgU6umCW0F4MDKox1h+tmlDaF+ahlA+Z6wpIZrp7j7oPDpRwf0JPtFyC7WkEVGsBvg+2jBkKiWI5k7d+9JTvy+MI3TQFrz2jE3tWehv1QXXuTA+qfkNP9QB/PbD0818ookh2nhUbjf3lJT9C7uWqS9dV3p5Dn71MPMV2/ufwVAR9CNM1xeHyC9/jidXNwGxmudBDe4Ze4DvGeaPgMMvt/6KPUf58nzwI0cjexo8N0wyjzpXikh6jC+RMETODJU6vktQmNqvrJi3HHJZy1qWbtmIcoWNHWr6lN4zkg2K+proJrt9G1xUx54WOUfdC5N0+jrhGRsLXJzRD9I20D5zERRPzv491J5OzOEy8Tk5bjxYYVrMmfIlVrJJNZ8sn4tQ9vmIAg3HHat6iayw4kJW3JdWLzTBgf16vQsIAGCwtsQe2ovRNBWNDEmeTyGQpXjimJ0pv4b54LdXjhvzKhf/6ZGfvzM5q4WnMPr/16IQigGhVnDU1v/eo29i4pZGR1/utmzFOAiGX+ASrPG6Mha9QP7DPIWaMTR4FE3k9/aoiqVsS5NJI+VdYyMIfVA9+uTd/tx9QZDiKZLQPsP0Zkz5bxy0Nb+hZzj3KZO+46xaRUrI= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3303.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230025)(39860400002)(396003)(346002)(376002)(366004)(136003)(451199018)(186003)(9686003)(26005)(76116006)(82960400001)(122000001)(38100700002)(83380400001)(38070700005)(7116003)(8936002)(9326002)(52536014)(8676002)(41300700001)(66476007)(66946007)(66556008)(2906002)(64756008)(66446008)(71200400001)(7696005)(55016003)(5660300002)(478600001)(6506007)(316002)(33656002)(86362001)(110136005)(41533002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?Windows-1252?Q?68M3BRLwoC3PybdhhWlc/xkrgWc6066EV6XbA8rDjTA0iiVzodEC8WlJ?= =?Windows-1252?Q?HysSfNWIdlGK2RoQozm4Mdt2pdq7Q+PlTljsBoPvnHq8fa05z506thiY?= =?Windows-1252?Q?1pAr94vzjTKoruzYN2EB7W8kvIgPwyXC9eHyaqH/p2ht01wwJ4eUu0S2?= =?Windows-1252?Q?Ad8SiV1Ss1cMTCeDR58OGxKlJJu5Fik7s+WEnaKQY686yGNIzWCfRsor?= =?Windows-1252?Q?dBMg/BN0GXkDmLHWRtbflqiw0y4CiiJwnnTepuO3iyeWMJ86F4boeyp4?= =?Windows-1252?Q?YyVcuZsVZU9MAa9B7tl64YlN4Bt+i4Oq0ffvQF0vT/qajD8gSCsj+14p?= =?Windows-1252?Q?xhZ+66MhKnOdRZbsojFDtvf2XWgTe4iW19Fi+Jey1PDKpIi2/c4AkcND?= =?Windows-1252?Q?PaMIlxNp7bNQ+CqlDoHb3LgbysqgIpC5v1Ah7DldeNPTGYNSR8oD6zJT?= =?Windows-1252?Q?Crca5SARhkGrzmhfPaNjXAiHU8+PPOoSqaguRo5kWdCtzKl3qIMriDih?= =?Windows-1252?Q?d4IpklYUinTCpEUi2gXoSRCbdULN7oqrhgsQLK60RDYiQ+xugIWs0OGF?= =?Windows-1252?Q?lx22/rEL+pPutGFdnlZJf2XTtJDFRjVc2CqtXESrA5hZlS4Sxn4DKrYD?= =?Windows-1252?Q?XIEta8UXdfUK6bb7XTyWCosnUGyOa4vT4HXMRI+x/tdc3lYQNz642qxD?= =?Windows-1252?Q?ls7O8aCO3BvXM9w03MY6nEoRPRDXOpMpnMdwHHh07Bqen0W6492Bb4HW?= =?Windows-1252?Q?CleCPR+4oY5fM30RgFLRIqSchCA/A+crfHvqFsyvc8GQ0fMa/Xixi+wD?= =?Windows-1252?Q?9RSEOIANpLWO6I84QHS2pw36nZxatMIsvPV6eAdj2RWYmMwP+IB19irV?= =?Windows-1252?Q?yvYfa05EJUfNzBY6qzavE2xO3FnRo5+YmBVOtfmMtmjSvsHvpWJNzBKz?= =?Windows-1252?Q?Zk6rtS4jZwTQpbnmyq0cJ92BHribBeqBuBnwLYEmzSIo/ZEbFGAPKRqY?= =?Windows-1252?Q?a0ZaI1pWIrRxK9HGtJcooYNxbyGfRrXTDK8hWCd8b49hHnjdnTDNa8PD?= =?Windows-1252?Q?vNWC7L259XFi3rRxs4XfuL2BuHPNDKnmdoqTTUjXI+/zBFv/IDsgIOVG?= =?Windows-1252?Q?Y4rLDsHB4eAOtVNZny1Z6srT2Ytw0heww/pwCWL/k+qFjoKuZmtWCDEI?= =?Windows-1252?Q?4P/qISB0RkAilWUm1oOH0+7TOdXa0HKUdeKo9er88F0Rrv4OrfRLKqez?= =?Windows-1252?Q?Z49kaUgL4u7XdHn1hq3bdwapg/V5n0NetYVrti+41b/7Y6qcZVftcV7A?= =?Windows-1252?Q?WVY3lqryPbqLtCSq6Sm6Zw2wT78M8HXOCUQ1XIjnAICXXwxlVedQxb17?= =?Windows-1252?Q?er+v0a70XHYFTknJLueEQkCGBfkXiaZ01CVEwThaFISzXWXdrWSU7Z6k?= =?Windows-1252?Q?0+YzPYlPTNARwWaR9DVYPW/B5frztR12WbGmPZkRJVtFwmfiv8o+DA5D?= =?Windows-1252?Q?WOoY8oTUVdSJMahD18eX3jJ9shvOhi0rKMvTDJip9vTJShP9PhSgSo7E?= =?Windows-1252?Q?C9HFn6jDVJYhlUPAY1PRXQC2zombNBp4ybSwAVcy09EPDFlhXkO0YAXn?= =?Windows-1252?Q?bNGUiFVd6LZjFnEp+t3eBvLhvg8N6nVGkbmKVYBGMUgqkFNtKwDxXrZv?= =?Windows-1252?Q?h+KpSZ/xMD5xkxsoIgs1g++7/vWjRTFGoHLCKcieqenUGJKZBw4qASQK?= =?Windows-1252?Q?nIWZzjo0XhhvRC6Un4A=3D?= Content-Type: multipart/alternative; boundary="_000_BYAPR11MB3303F73F9207639574F29B71DEA89BYAPR11MB3303namp_" MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3303.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: ee0b5e3f-7312-427c-5a4f-08db16139bcc X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Feb 2023 03:02:47.6795 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: XcSZE7KjbS9LyA7dPxP6Pgw6MFFhJMOWZF1dwJcSvW9LcskxYQqOneYjehlOhHFY0gV0hd/lXJjwXt8MKGitZxsg3wtWiWQOGtsH68K2Ewk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR11MB6343 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --_000_BYAPR11MB3303F73F9207639574F29B71DEA89BYAPR11MB3303namp_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Hi, I=92ve tracked down a regression in SPDK to DPDK commit a2833ecc5 (=93mempo= ol: fix get objects from mempool with cache=94). Here=92s an example that demonstrates the problem: Allocate mempool with 2048 buffers and cache size 256. Core 0 allocates 512 buffers. Mempool pulls 512 + 256 buffers from backing= ring, returns 512 of them to caller, puts the other 256 in core 0 cache. = Backing ring now has 1280 buffers. Core 1 allocates 512 buffers. Mempool pulls 512 + 256 buffers from backing= ring, returns 512 of them to caller, puts the other 256 in core 1 cache. = Backing ring now has 512 buffers. Core 2 allocates 512 buffers. Mempool pulls remaining 512 buffers from bac= king ring and returns all of them to caller. Backing ring now has 0 buffer= s. Core 3 tries to allocate 512 buffers and it fails. In the SPDK case, we don=92t really need or use the mempool cache in this c= ase, so changing the cache size to 0 fixes the problem and is what we=92re = going to move forward with. But the behavior did cause a regression so I thought I=92d mention it here.= If you have a mempool with 2048 objects, shouldn=92t 4 cores each be able= to do a 512 buffer bulk get, regardless of the configured cache size? Regards, Jim Harris --_000_BYAPR11MB3303F73F9207639574F29B71DEA89BYAPR11MB3303namp_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable

Hi,

 

I=92ve tracked down a regression in SPDK to DPDK com= mit a2833ecc5 (=93mempool: fix get objects from mempool with cache=94).

 

Here=92s an example that demonstrates the problem:

 

Allocate mempool with 2048 buffers and cache size 25= 6.

Core 0 allocates 512 buffers.  Mempool pulls 51= 2 + 256 buffers from backing ring, returns 512 of them to caller, puts the = other 256 in core 0 cache.  Backing ring now has 1280 buffers.

Core 1 allocates 512 buffers.  Mempool pulls 51= 2 + 256 buffers from backing ring, returns 512 of them to caller, puts the = other 256 in core 1 cache.  Backing ring now has 512 buffers.

Core 2 allocates 512 buffers.  Mempool pulls re= maining 512 buffers from backing ring and returns all of them to caller.&nb= sp; Backing ring now has 0 buffers.

Core 3 tries to allocate 512 buffers and it fails.

 

In the SPDK case, we don=92t really need or use the = mempool cache in this case, so changing the cache size to 0 fixes the probl= em and is what we=92re going to move forward with.


But the behavior did cause a regression so I thought I=92d mention it here.=   If you have a mempool with 2048 objects, shouldn=92t 4 cores each be= able to do a 512 buffer bulk get, regardless of the configured cache size?=

 

Regards,

 

Jim Harris

 

 

--_000_BYAPR11MB3303F73F9207639574F29B71DEA89BYAPR11MB3303namp_--