From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 46C1EA0093; Wed, 9 Nov 2022 23:46:02 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2A533400EF; Wed, 9 Nov 2022 23:46:02 +0100 (CET) Received: from EUR03-AM7-obe.outbound.protection.outlook.com (mail-am7eur03on2088.outbound.protection.outlook.com [40.107.105.88]) by mails.dpdk.org (Postfix) with ESMTP id 3BAD7400D4 for ; Wed, 9 Nov 2022 23:46:00 +0100 (CET) ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=byW3xhd1ZtQJXDspw2wR58ZrTviOZ955CcUgozG9U9AFOdJmz6UJmO//4Rk6F3mUS3S/XZa6zCIy8xaHB6eGypQ+v267kWwAyIxacypedblAJY6s39WnYU1bh+P4QM2rIiZPFvZsaJNmh0qGKBU5YzRhsNLp490KMI1aQ7BwKMocmQHqoMcs2mU1DtNII/nOdmfxeGKnp3+DELqLXXLi2z90ELqz/KsaDNMpDCE92dzv+pzBPuFGqgHiIGLLJs3zXQ1bIbh09YN3Ka4ED/x+CKb0NsS+WuVf74lk4VmiRtBVWlTy4//PLWrG4dMvZrZ5iXOcmQgU2KYWhBTFl4ZnRw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=K3Q0xyh5W+VeEzRF79Bq41Qfl3IvDLZwTDsluBJ/vPA=; b=IwR4Vibck7DMst+pIzZf5n09AKV49xeDZ+/940Lul0RBM2Iu1yxG3hFcTXBunLovbTigSmPteQeIEciPcENDrAsVxX6+HNss4jzc9yRhY8GN7+z13e9Vo5kkD8KeEacaCh3EIoppVQ6kaoI+xOO2ITnpRLzO0Qfcz0LMgCOxdTRi7+ksIfflO8vie4wic1SV9WAXC+YyD48xUrY+Z7X//OHTPb2sruY1FeCorXT613oFbuELonhn+pr19xGt/z4+l0DpgOCghwBkwiyM9POC2wU5s3C+5ywmZk28L7QPLHjNqE+D3b3mDkErfJXScheMLlh+Uv8+aI5hf/4WsnkBRg== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=dpdk.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K3Q0xyh5W+VeEzRF79Bq41Qfl3IvDLZwTDsluBJ/vPA=; b=S8k0/+N8Hly9yKQYkHb7LssxI3jv+aMWVNyteD6vqtT4K3knedS7dLzF3p++cipMsE2l4k8ntHZHTDxIUIMjK45gWydTEqcVOvhHjJup09cj2KqURyCHRPIYZ0cEsP/YdriOagc83dPHqKONXcHWUuVuV2Snz/5/CfuEBDoGo/M= Received: from AM6PR0202CA0061.eurprd02.prod.outlook.com (2603:10a6:20b:3a::38) by AS2PR08MB9341.eurprd08.prod.outlook.com (2603:10a6:20b:59a::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.12; Wed, 9 Nov 2022 22:45:51 +0000 Received: from AM7EUR03FT006.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:3a:cafe::bf) by AM6PR0202CA0061.outlook.office365.com (2603:10a6:20b:3a::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.22 via Frontend Transport; Wed, 9 Nov 2022 22:45:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM7EUR03FT006.mail.protection.outlook.com (100.127.141.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.12 via Frontend Transport; Wed, 9 Nov 2022 22:45:51 +0000 Received: ("Tessian outbound aeae1c7b66fd:v130"); Wed, 09 Nov 2022 22:45:51 +0000 X-CR-MTA-TID: 64aa7808 Received: from 6358e82e020f.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id BFEC5C9A-8942-4588-8A59-49D494176BBE.1; Wed, 09 Nov 2022 22:45:42 +0000 Received: from EUR03-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 6358e82e020f.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 09 Nov 2022 22:45:42 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=k9/wtBzlQYgnaCoJD4woWmxD1PTOIYsIt4/WhqRXCzdJmvIP2Uo0aW2NjBzpmIjyvr0MGWIkTcf0XIbYEzvA1Lhq5zRH/F1gEd9NnwUH4xQhoF+lENMEswZoz7mC6FxwMS8+YNEoRTh5tCer8YFbr66ImaBl/3MysrT8evCjH2yKE1ZH0S/qOEgbjnJYfi6ateG+fUluMm3OBCThfCz0YecKrl8a39JvdzdAymqZTLvtvoK+RZrz9WTZJqgjQrDWpjY50fg8ym8syOk53fdJRv8VKaChibzEFD+vWTG3LnSrR+oREMVlXprprjLmkepg83HczY4gdfugdyNeKfYPXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=K3Q0xyh5W+VeEzRF79Bq41Qfl3IvDLZwTDsluBJ/vPA=; b=hkcrrHBDPexI8cOA0STBDIr4I8NhPg6D/MMBqaqAf04VKeKHH9gv+oi2KKVnODpUJ83hByvuwvvcChu/6uH7y7Hns/2Mx978ip/5yIbSTqceRAhm4gQH4SKEikCpRAcgzOzikEe3xU2ouOi4FNd9S2cwg9Td5K6JfYX3Oz5jELuTEWQP4nII6XyMMjI4E19DcwOp194BV55f2wELONeQ4vx95N5VTRaqVRQsRg6HcH2Q7H8zopYD1XqCr0QkXQpI7F7N/h9CVZZnYbpuqOtw03Ff3FVnjFq8rl6SfxVOuW/PTcBUaqNSI0t85APQdR0goFqt2toe49eWa6s53UPgEQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K3Q0xyh5W+VeEzRF79Bq41Qfl3IvDLZwTDsluBJ/vPA=; b=S8k0/+N8Hly9yKQYkHb7LssxI3jv+aMWVNyteD6vqtT4K3knedS7dLzF3p++cipMsE2l4k8ntHZHTDxIUIMjK45gWydTEqcVOvhHjJup09cj2KqURyCHRPIYZ0cEsP/YdriOagc83dPHqKONXcHWUuVuV2Snz/5/CfuEBDoGo/M= Received: from DBAPR08MB5814.eurprd08.prod.outlook.com (2603:10a6:10:1b1::6) by AS2PR08MB10266.eurprd08.prod.outlook.com (2603:10a6:20b:62f::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.12; Wed, 9 Nov 2022 22:45:39 +0000 Received: from DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::3fb6:b7b2:1e8d:11d6]) by DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::3fb6:b7b2:1e8d:11d6%9]) with mapi id 15.20.5813.012; Wed, 9 Nov 2022 22:45:39 +0000 From: Honnappa Nagarahalli To: =?iso-8859-1?Q?Morten_Br=F8rup?= , "dev@dpdk.org" , "olivier.matz@6wind.com" , "andrew.rybchenko@oktetlabs.ru" , Kamalakshitha Aligeri , Bruce Richardson CC: nd , nd Subject: RE: [RFC] mempool: zero-copy cache put bulk Thread-Topic: [RFC] mempool: zero-copy cache put bulk Thread-Index: AdjxHCA/ROrKGJlPRZqF8NgKVJugtwAS1RvwABAiw0AArl49MAAB1pgQAAgXhFA= Date: Wed, 9 Nov 2022 22:45:39 +0000 Message-ID: References: <98CBD80474FA8B44BF855DF32C47DC35D87489@smartserver.smartshare.dk> <98CBD80474FA8B44BF855DF32C47DC35D8748A@smartserver.smartshare.dk> <98CBD80474FA8B44BF855DF32C47DC35D874A9@smartserver.smartshare.dk> In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D874A9@smartserver.smartshare.dk> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: C0822BA9CE03B148B940F368888E1959.0 x-checkrecipientchecked: true Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; x-ms-traffictypediagnostic: DBAPR08MB5814:EE_|AS2PR08MB10266:EE_|AM7EUR03FT006:EE_|AS2PR08MB9341:EE_ X-MS-Office365-Filtering-Correlation-Id: 28137935-b765-4f06-deae-08dac2a42762 x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: /cpg9qF4mV/Hwvi3iqyQp69u4KbcIeU6SwUONe8kgDWC2omD+5NCZJlRI1dcoCcrahlWsUcBtRKHJOMkq0Lxok3BUREInLg86WbgmjDUBYhEX9nvtijJAc2ZT8cPuQjssFun1cnlt3zmAhqkriYFZ/Drl62g5C6T949XJBCGIjV7/6e3/zSy4OfwsROlTWSNboHZKuly5pfsu7en/yxTsuDaI6ZDMC3JaywV3v7g+u1SBsnotIPZ6TRDZJdaiXA+MGyz0c14wJ7Hmo1I2d9Lz3bnlOZebutbfHqJFtR3GWIkZlvR/a4PJN84tFovOy0pV527Jvu3ybq8/fZoe1rF4a1bkET3XnDlTThsw15dZBUhCvcVL4CT8TsciDJ83xsyU8NrJHOZPjNS1G1RTGEelpe6F1+ahVasvel68z5D0YFGuLzDp2PcAsuF+b4ZPSfzPDR6dYabn3cxerVxdBWrvd44rhO8bCxr4SpVc21jH3jb2lqwf9QBPb/FuVjDG8rQJF0MTHD2R+Aoki5RFYOBoVhr84JoRPNRbvh5pKvpSYbt3yEHT5CQyoP9cPlY9hCp4eMi6zMC4vY3k7tQVpSPiWjAlWSJuju/BdOdg+cTypMQldZtpqrBtMLsKeyvwbVLC5LOvE3Z3XZkgDUAQolL5EO9/qaMKEeWdXmYHx6i4GyzH+Ywq1t6nAz9Wn1AmbqidhLYwt1rkpsjkZ98F2zGzwuYvRCWSJz6Oo11S2Q7i4aNn7MGu7vqPnniGyMd4qxYOu4pqX/05Z+1WvYGJhNKppYiSY6FceKsR07+B/krcrupUNmBhWkyaVjIdzN+zmzYdKQ7XZ0iehCZdY/LKIFa7g== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBAPR08MB5814.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(4636009)(366004)(346002)(376002)(136003)(396003)(39860400002)(451199015)(83380400001)(86362001)(966005)(33656002)(38100700002)(38070700005)(122000001)(478600001)(55016003)(66556008)(71200400001)(66946007)(5660300002)(66476007)(52536014)(76116006)(8936002)(316002)(54906003)(110136005)(4326008)(66446008)(8676002)(64756008)(41300700001)(9686003)(186003)(66574015)(26005)(30864003)(7696005)(2906002)(6506007)(84970400001)(66899015); DIR:OUT; SFP:1101; Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS2PR08MB10266 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM7EUR03FT006.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: c0bac6d3-984d-4f57-bbc8-08dac2a42002 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ap1jla53kwi1Mp0B7hAX/2wRTmemIPWFV/uuwCj4r/WZOW1IZ89M06Z1OatPbYD8Z0lpOQJHw6v560zzD/B1SQJQfpRMwXvhTMLzBppEs9vWCAXZApW0MzfOJg3nZkZtvPdiZUYx0Os9HspC8A0loE+JphRB08bQcVwMmuaeauc6RvGWL2HhJVWsp0eVIGQmwMRXlbweD+v67Pvf44bTxL/heeXYQNtQEgnQY3k7xN+/WBFdNO0wwP/4Y7lA2w1dCIfoAGY566G1KMAnAQl+oxuQqZ4o57ya0pLS26Kin6kia1kMRfWX4JBNR37AJQyR6LSYvfiL3mX3haMOr/r/LIBgre0ALsC/C/e01NGfNB5xXUi0nWCbWLj0mVlTXFJfr6UMoAez0MereyzHJiiu5724UJgHPKd0vI0IFbDYAB2PBBkzg2J3yCCCrBhGszkv0u+JW7pYkyJ/hQcsUPwvPCaTbA18z5cJOPlI/Da484QRNLUiyymGj99MxZIMFOmwTREa6auUOfTvU85bivCygyDlztUXaZZ28GkluBWmpm5gsOm1Xlv3hpi+nbrqwWCJw0bl/DWCs4J+ZHOASHr3mL5iob/Xj0f/i0Lr6gdbF2ohqLjenijuc8fFBPZJB308OM1e8SEaALOyiGElO7enmQ4ZG0XaNseCH4bRsE9mDdIpQLiH91Tv1yWJ2bC3UE92TIQEqIDGIUsdkn/uvjF2xC6jmciMrHbrZ3gBMXdxlDZPCdz55/JLXcRWFG1RdpvA88ZAbt0kUkJWwRrrcxhi+f+eXtNM9sAhrLwXyt4TAYRnddVizQeUC/0IlUIObf51Gp6L0+V8En2btFu/Ov61kw== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230022)(4636009)(346002)(39860400002)(136003)(396003)(376002)(451199015)(40470700004)(36840700001)(46966006)(70586007)(8676002)(82740400003)(70206006)(36860700001)(4326008)(316002)(55016003)(7696005)(40480700001)(336012)(47076005)(83380400001)(6506007)(40460700003)(33656002)(2906002)(66574015)(41300700001)(52536014)(86362001)(9686003)(26005)(8936002)(5660300002)(30864003)(186003)(66899015)(82310400005)(966005)(478600001)(54906003)(81166007)(356005)(110136005)(84970400001); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Nov 2022 22:45:51.6781 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 28137935-b765-4f06-deae-08dac2a42762 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM7EUR03FT006.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS2PR08MB9341 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org >=20 > +To: Bruce also showed interest in this topic, and might have more insigh= ts. >=20 > > From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] > > Sent: Wednesday, 9 November 2022 18.58 > > > > > > > > > > > > > From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] > > > > Sent: Sunday, 6 November 2022 00.11 > > > > > > > > + Akshitha, she is working on similar patch > > > > > > > > Few comments inline > > > > > > > > > From: Morten Br=F8rup > > > > > Sent: Saturday, November 5, 2022 8:40 AM > > > > > > > > > > Zero-copy access to the mempool cache is beneficial for PMD > > > > performance, > > > > > and must be provided by the mempool library to fix [Bug 1052] > > > > > without > > > > a > > > > > performance regression. > > > > > > > > > > [Bug 1052]: https://bugs.dpdk.org/show_bug.cgi?id=3D1052 > > > > > > > > > > > > > > > This RFC offers a conceptual zero-copy put function, where the > > > > application > > > > > promises to store some objects, and in return gets an address > > where > > > > to store > > > > > them. > > > > > > > > > > I would like some early feedback. > > > > > > > > > > Notes: > > > > > * Allowing the 'cache' parameter to be NULL, and getting it from > > the > > > > > mempool instead, was inspired by rte_mempool_cache_flush(). > > > > I am not sure why the 'cache' parameter is required for this API. > > This > > > > API should take the mem pool as the parameter. > > > > > > > > We have based our API on 'rte_mempool_do_generic_put' and removed > > > the > > > > 'cache' parameter. > > > > > > I thoroughly considered omitting the 'cache' parameter, but included > > it for > > > two reasons: > > > > > > 1. The function is a "mempool cache" function (i.e. primarily > > > working > > on the > > > mempool cache), not a "mempool" function. > > > > > > So it is appropriate to have a pointer directly to the structure it > > is working on. > > > Following this through, I also made 'cache' the first parameter and > > 'mp' the > > > second, like in rte_mempool_cache_flush(). > > I am wondering if the PMD should be aware of the cache or not. For ex: > > in the case of pipeline mode, the RX and TX side of the PMD are > > running on different cores. >=20 > In that example, the PMD can store two cache pointers, one for each of th= e > RX and TX side. I did not understand this. If RX core and TX core have their own per-core c= aches the logic would not work. For ex: the RX core cache would not get fil= led. In the case of pipeline mode, there will not be a per-core cache. The buffe= rs would be allocated and freed from a global ring or a global lockless sta= ck. >=20 > And if the PMD is unaware of the cache pointer, it can look it up at runt= ime > using rte_lcore_id(), like it does in the current Intel PMDs. >=20 > > However, since the rte_mempool_cache_flush API is provided, may be > > that decision is already done? Interestingly, rte_mempool_cache_flush > > is called by just a single PMD. >=20 > I intentionally aligned this RFC with rte_mempool_cache_flush() to mainta= in > consistency. >=20 > However, the API is not set in stone. It should always be acceptable to > consider improved alternatives. >=20 > > > > So, the question is, should we allow zero-copy only for per-core cache > > or for other cases as well. >=20 > I suppose that the mempool library was designed to have a mempool > associated with exactly one mempool cache per core. (Alternatively, the > mempool can be configured with no mempool caches at all.) >=20 > We should probably stay loyal to that design concept, and only allow zero= - > copy for per-core cache. >=20 > If you can come up with an example of the opposite, I would like to explo= re > that option too... I can't think of a good example myself, and perhaps I'= m > overlooking a relevant use case. The use case I am talking about is the pipeline mode as I mentioned above. = Let me know if you agree. >=20 > > > > > > > > 2. In most cases, the function only accesses the mempool structure > > > in > > order to > > > get the cache pointer. Skipping this step improves performance. > > > > > > And since the cache is created along with the mempool itself (and > > thus never > > > changes for a mempool), it would be safe for the PMD to store the > > 'cache' > > > pointer along with the 'mp' pointer in the PMD's queue structure. > > Agreed > > > > > > > > E.g. in the i40e PMD the i40e_rx_queue structure could include a > > "struct > > > rte_mempool_cache *cache" field, which could be used > > > i40e_rxq_rearm() > > [1] > > > instead of "cache =3D rte_mempool_default_cache(rxq->mp, > > rte_lcore_id())". > > > > > > [1] https://elixir.bootlin.com/dpdk/v22.11- > > > rc2/source/drivers/net/i40e/i40e_rxtx_vec_avx512.c#L31 > > > > > > > This new API, on success, returns the pointer to memory where the > > > > objects are copied. On failure it returns NULL and the caller has > > to > > > > call 'rte_mempool_ops_enqueue_bulk'. Alternatively, the new API > > could > > > > do this as well and PMD does not need to do anything if it gets a > > NULL > > > > pointer. > > > > > > Yes, we agree about these two details: > > > > > > 1. The function should return a pointer, not an integer. > > > It would be a waste to use a another CPU register to convey a > > success/error > > > integer value, when the success/failure information is just as > > > easily > > conveyed > > > by the pointer return value (non-NULL/NULL), and rte_errno for > > various error > > > values in the unlikely cases. > > > > > > 2. The function should leave it up to the PMD what to do if direct > > access to > > > the cache is unavailable. > > Just wondering about the advantage of this. I do not think PMD's have > > much of a choice other than calling 'rte_mempool_ops_enqueue_bulk' >=20 > I agree, but that was not my point. Let me try to rephrase: >=20 > The PMD is more likely to know how to efficiently build the array of mbuf= s to > pass to rte_mempool_ops_enqueue_bulk() than the mempool library - many > PMDs already implement a variety of vector instruction variants to do exa= ctly > that. So we should not try to be clever and add a fallback path - this jo= b > belongs in the PMD. >=20 > The PMD might not even have the array of mbufs lined up when calling > rte_mempool_cache_put_bulk_promise(). The PMD could have an array of > internal structures, where the mbuf pointer is an element in that structu= re. Agree, you are correct. We should leave it to PMD to handle the failure cas= e. >=20 > > > > > > > > > > > > > We should think about providing similar API on the RX side to > > > > keep > > it > > > > symmetric. > > > > > > I sent an RFC for that too: > > > > http://inbox.dpdk.org/dev/98CBD80474FA8B44BF855DF32C47DC35D87488@ > > > smartserver.smartshare.dk/T/#u > > > > > > > > > > > > > > > * Asserting that the 'mp' parameter is not NULL is not done by > > other > > > > > functions, so I omitted it here too. > > > > > > > > > > NB: Please ignore formatting. Also, this code has not even been > > > > compile > > > > > tested. > > > > We are little bit ahead, tested the changes with i40e PF PMD, > > > > wrote unit test cases, going through internal review, will send > > > > out RFC > > on > > > > Monday > > > > > > Sounds good. Looking forward to review. > > > > > > > > > > > > > > > > > /** > > > > > * Promise to put objects in a mempool via zero-copy access to a > > > > user-owned > > > > > mempool cache. > > > > > * > > > > > * @param cache > > > > > * A pointer to the mempool cache. > > > > > * @param mp > > > > > * A pointer to the mempool. > > > > > * @param n > > > > > * The number of objects to be put in the mempool cache. > > > > > * @return > > > > > * The pointer to where to put the objects in the mempool > > cache. > > > > > * NULL on error > > > > > * with rte_errno set appropriately. > > > > > */ > > > > > static __rte_always_inline void * > > > > > rte_mempool_cache_put_bulk_promise(struct rte_mempool_cache > > > *cache, > > > > > struct rte_mempool *mp, > > > > > unsigned int n) > > > > > { > > > > > void **cache_objs; > > > > > > > > > > if (cache =3D=3D NULL) > > > > > cache =3D rte_mempool_default_cache(mp, rte_lcore_id()); > > Any reason we need this? If we are expecting the PMD to store the > > pointer to cache and a NULL is passed, it would mean it is a mempool > > with no per-core cache? > > We could also leave the NULL check to the PMD. >=20 > Personally, I would strongly prefer requiring the cache pointer to be val= id, > and use RTE_ASSERT() here, instead of allowing a NULL pointer as a specia= l > case to look it up inside the function. I consider this special NULL case= useless > bloat, which does not belong in a fast path library function. >=20 > But I copied this approach from rte_mempool_cache_flush(). The API definition does not bind it to do this check. We might be able to d= elete the check in rte_mempool_cache_flush. >=20 > We could expose an "unsafe" function where is not allowed to pass NULL > pointers, and a "safe" function (fixing the cache pointer if NULL) for > consistency. >=20 > If the rte_mempool_cache_flush() function is popular, we could also expos= e > an "unsafe" variant where passing NULL pointers are disallowed. >=20 > I wonder if there are any examples of such safe/unsafe variants in DPDK? = It > would be nice with a common naming convention for such function variants. >=20 > > > > > > > if (cache =3D=3D NULL) { > > > > > rte_errno =3D EINVAL; > > > > > return NULL; > > > > > } > > > > > > > > > > rte_mempool_trace_cache_put_bulk_promise(cache, mp, n); > > > > > > > > > > /* The request itself is too big for the cache */ > > > > > if (unlikely(n > cache->flushthresh)) { > > > > > rte_errno =3D EINVAL; > > > > > return NULL; > > > > > } > > > > > > > > > > /* > > > > > * The cache follows the following algorithm: > > > > > * 1. If the objects cannot be added to the cache without > > > > crossing > > > > > * the flush threshold, flush the cache to the backend. > > > > > * 2. Add the objects to the cache. > > > > > */ > > > > > > > > > > if (cache->len + n <=3D cache->flushthresh) { > > > > > cache_objs =3D &cache->objs[cache->len]; > > > > > cache->len +=3D n; > > > > > } else { > > > > > cache_objs =3D &cache->objs[0]; > > > > > rte_mempool_ops_enqueue_bulk(mp, cache_objs, cache->len); > > > > > cache->len =3D n; > > > > > } > > > > > > > > > > RTE_MEMPOOL_CACHE_STAT_ADD(cache, put_bulk, 1); > > > > > RTE_MEMPOOL_CACHE_STAT_ADD(cache, put_objs, n); > > These are new stats. Do these break ABI compatibility (though these > > are under DEBUG flag)? >=20 > They are not mempool cache stats, they are only kept in the cache structu= re > to provide alternative (i.e. faster) update access to some (i.e. the most= often > updated) of the existing mempool stats. The patch is [1], and part of a s= eries > currently being discussed if should go into 22.11-rc3 or not [2]. >=20 > [1]: > https://patchwork.dpdk.org/project/dpdk/patch/20221109181852.109856-3- > mb@smartsharesystems.com/ > [2]: > http://inbox.dpdk.org/dev/98CBD80474FA8B44BF855DF32C47DC35D874A6 > @smartserver.smartshare.dk/T/#m41bf4e8bd886db49f11c8dbd63741b35327 > 7082f >=20 > > > > > > > > > > > > return cache_objs; > > > > > } > > > > > > > > > > > > > > > Med venlig hilsen / Kind regards, -Morten Br=F8rup > > > > > > > > > > >