From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AB717A04FD; Thu, 10 Nov 2022 12:01:18 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9DBB140150; Thu, 10 Nov 2022 12:01:18 +0100 (CET) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by mails.dpdk.org (Postfix) with ESMTP id 85871400EF for ; Thu, 10 Nov 2022 12:01:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1668078077; x=1699614077; h=date:from:to:cc:subject:message-id:references: content-transfer-encoding:in-reply-to:mime-version; bh=w0McMtVUl34wSd0t7gYuBZ8CEgF5yu7xWOXsIx2Em2g=; b=Lh/s4Lhwis22qTqvMAbEFiT0ROqE9FmMrfTelxYQ1/SnQ/hKM10FDLVs x37u4L5EC01lBuGvmZ/+nl5wt/SXsecVtyO6pO3eONOPSKiIDfP6YSUIf mo09fBjyO0hVA5ub8w4Ct1bQpYlFKNlLeh28X+dBFxhZlhDEEEFP2atWb QV1p1MoIXJDI9+59k+oPgCPmWu0+pUY2Fg4/HpY6zlLVArG4LRmoYOk7s lkQdYeYuMWOR593NfKNv4h5jijjUoMwShvemvpbz57fyqPlsdiWF2LyHA hqyPYYVBfhMT+KrdvHVM409wcrEGJqvK7zv7kfcWR+Z5dcAM3VaBOapZK Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="298778074" X-IronPort-AV: E=Sophos;i="5.96,153,1665471600"; d="scan'208";a="298778074" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Nov 2022 03:01:16 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10526"; a="588135954" X-IronPort-AV: E=Sophos;i="5.96,153,1665471600"; d="scan'208";a="588135954" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by orsmga003.jf.intel.com with ESMTP; 10 Nov 2022 03:01:16 -0800 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 10 Nov 2022 03:01:16 -0800 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Thu, 10 Nov 2022 03:01:15 -0800 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31 via Frontend Transport; Thu, 10 Nov 2022 03:01:15 -0800 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.43) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.31; Thu, 10 Nov 2022 03:00:59 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fHHLq6tYkiqf9hLbmdn63xvMzWAhciObggpUOFtNp8uKthueclx7/YerMyd92ltnuF0N7GwmiDoP3haQzMOWeesQ26pYv24bP5y/dUdkcvEzPKfcCqOhrov2jnJjRl9kWPUQDSS40L3gKkW5Vg63SSgBkeSMkNfIsdmLzAAtfqMFARfz1BxE/YdifqFUeycrK5UaYzL/ZaBjQmuq/F05+bfAyt4c2LGoa3nrFTJzJKVu/OPreRCZzWpTxoQs1SlxfpNFaODOJRo6mY2mbu7n1h/yKQweGkimIFvAZRYJ1drxG1AQ3brGidSNiDxeo+HOHsrXRfxNyoJ+EU/F3qK15Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2JCDWhcgyR6kXxIt/z9CAW5uuh9kTIlx/0bdXwd/T58=; b=I7rgi9Kx7eDtmAEIFqCBe/hSYcEx2X4JqyTzSp3wgXfy1c2CmD4LKjVd/dJFtaIb5TC2mezsiWN7rgpgk8CgTlN3U2ncQ4dO8Wcio658Owt7k5AlGgAwWXG8kwwTidfYd6HzPWJVntE6/HH6mXS6mkjC5jApDCKvEFOluQQ5xgDsq6pfntOQASrgNyeyJ1E1hIcnU+Sq6EV01ooJ96LwuT4LCd4K/mi1sEMh4VW2HJYUPDujLLtJhE0Cq2igBuZ4uiRzPVh+5exjuDcv7Yd8R2ZcEgLiTChdc7j46GUHoSK26OcSrh7wFJI0OZ6FWBcmrC/xdtROKyAPfo+NxwZ6GQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from MWHPR11MB1629.namprd11.prod.outlook.com (2603:10b6:301:d::21) by BN9PR11MB5420.namprd11.prod.outlook.com (2603:10b6:408:101::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.27; Thu, 10 Nov 2022 11:00:50 +0000 Received: from MWHPR11MB1629.namprd11.prod.outlook.com ([fe80::d136:9b16:f5d7:30e3]) by MWHPR11MB1629.namprd11.prod.outlook.com ([fe80::d136:9b16:f5d7:30e3%3]) with mapi id 15.20.5813.013; Thu, 10 Nov 2022 11:00:50 +0000 Date: Thu, 10 Nov 2022 11:00:42 +0000 From: Bruce Richardson To: Morten =?iso-8859-1?Q?Br=F8rup?= CC: Honnappa Nagarahalli , , , , "Kamalakshitha Aligeri" , nd Subject: Re: [RFC] mempool: zero-copy cache put bulk Message-ID: References: <98CBD80474FA8B44BF855DF32C47DC35D87489@smartserver.smartshare.dk> <98CBD80474FA8B44BF855DF32C47DC35D8748A@smartserver.smartshare.dk> <98CBD80474FA8B44BF855DF32C47DC35D874A9@smartserver.smartshare.dk> <98CBD80474FA8B44BF855DF32C47DC35D874AB@smartserver.smartshare.dk> Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D874AB@smartserver.smartshare.dk> X-ClientProxiedBy: LO6P123CA0039.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:2fe::19) To MWHPR11MB1629.namprd11.prod.outlook.com (2603:10b6:301:d::21) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWHPR11MB1629:EE_|BN9PR11MB5420:EE_ X-MS-Office365-Filtering-Correlation-Id: 8b0ef579-455e-4d98-c0d8-08dac30ad35b X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zYtt4mq9I47p6aEdZeacFdvd3pY6ELH1NJCLG2hS9ONaC3TTBM79/gqt+Ks6QA5/bcOZVkVCTH1emi71zVEczawimbf2fO05rNwLFt6lFdodU7ebx9zJ9H+3tTHAX9AwGjfPPBr2bsH1NGsN8TNjZo/xlQkQu7rYnr49Hfoe5bx2JHwW9hEITUtVCwfHgN5rqZKG6uaxxGCOaymnjIbqXcaFlTEgExpGVNd+P+aCYjSyyidSZIzWkSVkPA8dffNme6YpEWUTGE7ki2nSCHAGpeXFfKlevrlIThRBzedkTKNz/BwYz3A1PnNS8O36q00fIEgXGRNVTAGZ93o1r07UTOCwCPyyy8ogcRtYGvIBwx8SlPnAAfKESrwc6u2af8cXwtB0V5YbKK/KkZNNP1PG+1COOdz4gMeyFL+jZHZrJN3pPsdv3lzkBqBSgERmKdqGzaUpCTdHS3QipqBP48uTbFwxGBO/Q8O885iKINNDIcecNK16MwExfL2WdXJQtQPORReosmBzI8O2+u2odzwC3vsd59reVZSNfJgsIQ2wVlqtq+XHAAscdUzTRP/GMszxxzP7igI+2jrfpo4AJVNuq6UkDxCoeu0V3f/6TlMIbtMh/32zHPkPtiguTI3zjil1tN4/1wZR5G52CoFM/RYBlKjPFtFWaM0LFBzlWdoh9N9Xm15dlA11NktscxHH6GJuW5kGsDzeaHbl0Pwnrvu04iXeAwiwosuyOn9kZIYkzC7nF2w6r6MCcgEXmyk4L228sKSzQNgle5IZZPXB7b+cyQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MWHPR11MB1629.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230022)(376002)(39860400002)(346002)(396003)(136003)(366004)(451199015)(66574015)(316002)(2906002)(6486002)(44832011)(30864003)(6512007)(26005)(8676002)(4326008)(966005)(41300700001)(8936002)(6666004)(5660300002)(6506007)(66476007)(66556008)(186003)(478600001)(86362001)(66946007)(66899015)(38100700002)(82960400001)(6916009)(54906003)(83380400001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?iso-8859-1?Q?TY+Fa21P2BYEs4dW8GcKLnGPBn5bNb3mMfl9s/6Ktdcid5xST9+bPWilEW?= =?iso-8859-1?Q?9VNkfrpOsBAqPAw5oxBgJinfPdQX21DTcLGHNbtMuujvq11TrE1WF3qONp?= =?iso-8859-1?Q?gT7bM5YhGFAFDxlcXDRVFVm1pgqYC9GIA9yGe0owisH32Vo7wxd++WoWZl?= =?iso-8859-1?Q?0qiuQk01bqHtxjdYNN9iQIa9N3YfFYxjcrwbteHQpOJsbLiMt9HpO1w9Hd?= =?iso-8859-1?Q?9MtO4PvpqS+jQzaxcdRcdK7IBsr/GTLWFqP6r+r/EVI6kGIJMxnZhl2UYQ?= =?iso-8859-1?Q?NZgHRbFEie1XXZjjl1rWryGwXqAPL+LhYUszTMpqvjQqbt3xVwxfx8uXIV?= =?iso-8859-1?Q?hP4btSTPny/p051vI6KncqIHiT0MC78L/mcpAe4k9ltbAPwU9Ba6GGJWbh?= =?iso-8859-1?Q?0h086YSxHt4ECrT0cj1DYpQs74ZWVzJ9XLFOs/PiEvBVefj6+dx2iyYOgK?= =?iso-8859-1?Q?9sQJECDQ37zYYbV/2yqaa4ZGR2jxBvLLuvbNE0JofTUB0YPuwqFz9zBf1l?= =?iso-8859-1?Q?OPwtwBMAx+RBwr7ADjtetxZPTUMRxXjU6m99ep9Lpex2HlGjq5kRzVqHvj?= =?iso-8859-1?Q?+rt8cfPio0SqhXHXLu4oPyhAS8spOLOs5XSQADtH+fzs6CZnDDpCO4QnE0?= =?iso-8859-1?Q?1jAETvFKHc1WhREydP+RI47TQBNzliA9PHAsw0uIPd43B2xtSjI59yu6Kb?= =?iso-8859-1?Q?ch2SemYqT+XCa+ALGDn9cAUsqVGlpNE8QAIlSZ5GynZGRAhrKEq8Oi3rEU?= =?iso-8859-1?Q?yKhAXgDhSUFFpxguDO+R1bSTA3qSctq30lPB2JWWN3TCJIdv/MUukd330Z?= =?iso-8859-1?Q?U2Sf+T3pk7dd0a6Nf2KCohhAAIGmHYQ3IEN6rrU7hA65a6022GRaa0vB+X?= =?iso-8859-1?Q?9jSvmBkppnTGCqS5dvINcFDnLI0ve2cIwELI3DXQ1206QYfGHf+SSIMgNh?= =?iso-8859-1?Q?UCTX5j4GpO6ET2eA0pjfsINgiSYjr3fhyD0/wa3r5Ez3NKsKnc9DYegDd5?= =?iso-8859-1?Q?KnPgyE+Xvnd5tQBzZtwyckpuPA67BJ6/tlKo7ZVubEfWd++sV6sWF753dO?= =?iso-8859-1?Q?N4+MHq/IYsi6Ry0Vm9E7C0TnOyWbhaR+FCdH1ueGN8ivTn0qY8yUVCLH3m?= =?iso-8859-1?Q?RuSdmd5NI/aMxvOi+VrHUc8EmxwQg6aYbSEscuk1opS7yuFcc8ZjNRdaWO?= =?iso-8859-1?Q?B6NV2Qtcy6noSfrntI6OLR6ld14f8lxfC60IpYk6bfu2kswgmEe9KMxMiT?= =?iso-8859-1?Q?iprb5kvFJP1ZDBV94uB2VXwsWm26PoH8KDb2rFmrv/KpRZVE2lLRAWJVbM?= =?iso-8859-1?Q?X9DIfS6MRirMIJzZ21WqtTqpBQxbp8X2hl9R5+FlRs8zs+txRUaj3fRQr6?= =?iso-8859-1?Q?n9J4G/RZBuuetpCxIO+4+UHsxx2M5HUKJGc/OUwS9WEzSCailVHbJSSxvn?= =?iso-8859-1?Q?/OrDqYtQxUSKhssXoWQ77iehGKJzLIlsNaQ/WliKHUY93hhjwaFZIkldpe?= =?iso-8859-1?Q?N8qLQ510xt+wLwn6y5eN9t6Y5irkxG1wRGzU5w/bMcKQry1MlB2S2Xu+qv?= =?iso-8859-1?Q?o0ocSh+VSGlMtQ8ytz97x21jO6dXdpD4R7MYRdQIR9zUGGnxy7BD20OF+n?= =?iso-8859-1?Q?GwCmUEQKHJ1XeGIgKman1SxuV2yKebgsW19IcwOrkWUkRIVCxM0cfvsg?= =?iso-8859-1?Q?=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: 8b0ef579-455e-4d98-c0d8-08dac30ad35b X-MS-Exchange-CrossTenant-AuthSource: MWHPR11MB1629.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2022 11:00:50.5109 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: xNjJjMQNZ+HIO79d0RWuG1zqjPDwL613F2Y9wYZqkDZ/9np5WhKNagmfQD8+UMwbzueEiEBW44us9hwG18U6DNe4G9Oxvg54GGTZoVNoMhE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR11MB5420 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Thu, Nov 10, 2022 at 11:15:27AM +0100, Morten Brørup wrote: > > From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] > > Sent: Wednesday, 9 November 2022 23.46 > > > > > > +To: Bruce also showed interest in this topic, and might have more > > insights. > > > > > > > From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] > > > > Sent: Wednesday, 9 November 2022 18.58 > > > > > > > > > > > > > > > > > > > > > > > From: Honnappa Nagarahalli > > [mailto:Honnappa.Nagarahalli@arm.com] > > > > > > Sent: Sunday, 6 November 2022 00.11 > > > > > > > > > > > > + Akshitha, she is working on similar patch > > > > > > > > > > > > Few comments inline > > > > > > > > > > > > > From: Morten Brørup > > > > > > > Sent: Saturday, November 5, 2022 8:40 AM > > > > > > > > > > > > > > Zero-copy access to the mempool cache is beneficial for PMD > > > > > > performance, > > > > > > > and must be provided by the mempool library to fix [Bug 1052] > > > > > > > without > > > > > > a > > > > > > > performance regression. > > > > > > > > > > > > > > [Bug 1052]: https://bugs.dpdk.org/show_bug.cgi?id=1052 > > > > > > > > > > > > > > > > > > > > > This RFC offers a conceptual zero-copy put function, where > > the > > > > > > application > > > > > > > promises to store some objects, and in return gets an address > > > > where > > > > > > to store > > > > > > > them. > > > > > > > > > > > > > > I would like some early feedback. > > > > > > > > > > > > > > Notes: > > > > > > > * Allowing the 'cache' parameter to be NULL, and getting it > > from > > > > the > > > > > > > mempool instead, was inspired by rte_mempool_cache_flush(). > > > > > > I am not sure why the 'cache' parameter is required for this > > API. > > > > This > > > > > > API should take the mem pool as the parameter. > > > > > > > > > > > > We have based our API on 'rte_mempool_do_generic_put' and > > removed > > > > > the > > > > > > 'cache' parameter. > > > > > > > > > > I thoroughly considered omitting the 'cache' parameter, but > > included > > > > it for > > > > > two reasons: > > > > > > > > > > 1. The function is a "mempool cache" function (i.e. primarily > > > > > working > > > > on the > > > > > mempool cache), not a "mempool" function. > > > > > > > > > > So it is appropriate to have a pointer directly to the structure > > it > > > > is working on. > > > > > Following this through, I also made 'cache' the first parameter > > and > > > > 'mp' the > > > > > second, like in rte_mempool_cache_flush(). > > > > I am wondering if the PMD should be aware of the cache or not. For > > ex: > > > > in the case of pipeline mode, the RX and TX side of the PMD are > > > > running on different cores. > > > > > > In that example, the PMD can store two cache pointers, one for each > > of the > > > RX and TX side. > > I did not understand this. If RX core and TX core have their own per- > > core caches the logic would not work. For ex: the RX core cache would > > not get filled. > > > > In the case of pipeline mode, there will not be a per-core cache. The > > buffers would be allocated and freed from a global ring or a global > > lockless stack. > > Aha... Now I understand what you mean: You are referring to use cases where the mempool is configured to *not* have a mempool cache. > > For a mempool without a mempool cache, the proposed "mempool cache" zero-copy functions can obviously not be used. > > We need "mempool" zero-copy functions for the mempools that have no mempool cache. > > However, those functions depend on the mempool's underlying backing store. > > E.g. zero-copy access to a ring has certain requirements [1]. > > [1]: http://doc.dpdk.org/guides/prog_guide/ring_lib.html#ring-peek-zero-copy-api > > For a stack, I think it is possible to locklessly zero-copy pop objects. But it is impossible to locklessly zero-copy push elements to a stack; another thread can race to pop some objects from the stack before the pushing thread has finished writing them into the stack. > > Furthermore, the ring zero-copy get function cannot return a consecutive array of objects when wrapping, and PMD functions using vector instructions usually rely on handling chunks of e.g. 8 objects. > > Just for a second, let me theorize into the absurd: Even worse, if a mempool's underlying backing store does not use an array of pointers as its internal storage structure, it is impossible to use a pointer to an array of pointers for zero-copy transactions. E.g. if the backing store uses a list or a tree structure for its storage, a pointer to somewhere in the list or tree structure is not an array of objects pointers. > > Anyway, we could consider designing a generic API for zero-copy mempool get/put; but it should be compatible with all underlying backing stores - or return failure, so the PMD can fall back to the standard functions, if the mempool is in a state where zero-copy access to a contiguous burst cannot be provided. E.g. zero-copy get from a ring can return failure when zero-copy access to the ring is temporarily unavailable due to being at a point where it would wrap. > > Here is a conceptual proposal for such an API. > > /* Mempool zero-copy transaction state. Opaque outside the mempool API. */ > struct rte_mempool_zc_transaction_state { > char opaque[RTE_CACHE_LINE_SIZE]; > }; > > /** Start zero-copy get/put bulk transaction. > * > * @param[in] mp > * Pointer to the mempool. > * @param[out] obj_table_ptr > * Where to store the pointer to > * the zero-copy array of objects in the mempool. > * @param[in] n > * Number of objects in the transaction. > * @param[in] cache > * Pointer to the mempool cache. May be NULL if unknown. > * @param[out] transaction > * Where to store the opaque transaction information. > * Used internally by the mempool library. > * @return > * - 1: Transaction completed; > * '_finish' must not be called. > * - 0: Transaction started; > * '_finish' must be called to complete the transaction. > * - <0: Error; failure code. > */ > static __rte_always_inline int > rte_mempool_get/put_zc_bulk_start( > struct rte_mempool *mp, > void ***obj_table_ptr, > unsigned int n, > struct rte_mempool_cache *cache, > rte_mempool_zc_transaction_state *transaction); > > /** Finish zero-copy get/put bulk transaction. > * > * @param[in] mp > * Pointer to mempool. > * @param[in] obj_table_ptr > * Pointer to the zero-copy array of objects in the mempool, > * returned by the 'start' function. > * @param[in] n > * Number of objects in the transaction. > * Must be the same as for the 'start' function. > * @param[in] transaction > * Opaque transaction information, > * returned by the 'start' function. > * Used internally by the mempool library. > */ > static __rte_always_inline void > rte_mempool_get/put_zc_bulk_finish( > struct rte_mempool *mp, > void **obj_table, > unsigned int n, > rte_mempool_zc_transaction_state *transaction); > > Note that these are *bulk* functions, so 'n' has to remain the same for a 'finish' call as it was for the 'start' call of a transaction. > > And then the underlying backing stores would need to provide callbacks that implement these functions, if they offer zero-copy functionality. > > The mempool implementation of these could start by checking for a mempool cache, and use the "mempool cache" zero-copy if present. > > Some internal state information (from the mempool library or the underlying mempool backing store) may need to be carried over from the 'start' to the 'finish' function, so I have added a transaction state parameter. The transaction state must be held by the application for thread safety reasons. Think of this like the 'flags' parameter to the Linux kernel's spin_lock_irqsave/irqrestore() functions. > > We could omit the 'obj_table' and 'n' parameters from the 'finish' functions and store them in the transaction state if needed; but we might possibly achieve higher performance by passing them as parameters instead. > > > > > > > > > And if the PMD is unaware of the cache pointer, it can look it up at > > runtime > > > using rte_lcore_id(), like it does in the current Intel PMDs. > > > > > > > However, since the rte_mempool_cache_flush API is provided, may be > > > > that decision is already done? Interestingly, > > rte_mempool_cache_flush > > > > is called by just a single PMD. > > > > > > I intentionally aligned this RFC with rte_mempool_cache_flush() to > > maintain > > > consistency. > > > > > > However, the API is not set in stone. It should always be acceptable > > to > > > consider improved alternatives. > > > > > > > > > > > So, the question is, should we allow zero-copy only for per-core > > cache > > > > or for other cases as well. > > > > > > I suppose that the mempool library was designed to have a mempool > > > associated with exactly one mempool cache per core. (Alternatively, > > the > > > mempool can be configured with no mempool caches at all.) > > > > > > We should probably stay loyal to that design concept, and only allow > > zero- > > > copy for per-core cache. > > > > > > If you can come up with an example of the opposite, I would like to > > explore > > > that option too... I can't think of a good example myself, and > > perhaps I'm > > > overlooking a relevant use case. > > The use case I am talking about is the pipeline mode as I mentioned > > above. Let me know if you agree. > > I see what you mean, and I don't object. :-) > > However, I still think the "mempool cache" zero-copy functions could be useful. > > They would be needed for the generic "mempool" zero-copy functions anyway. > > And the "mempool cache" zero-copy functions are much simpler to design, implement and use than the "mempool" zero-copy functions, so it is a good first step. > I would think that even in pipeline mode applications a mempool cache would still be very useful, as it would like reduce the number of accesses to the underlying shared ring or stack resources. For example, if, as is common, the application works in bursts of up to 32, but the mempool cache was configured as 128, it means that the TX side would flush buffers to the shared pool at most every 4 bursts and likely less frequently than that due to the bursts themselves not always being the full 32. Since accesses to the underlying ring and stack tend to be slow due to locking or atomic operations, the more you can reduce the accesses the better. Therefore, I would very much consider use of a mempool without a cache as an edge-case - but one we need to support, but not optimize for, since mempool accesses without cache would already be rather slow. My 2c here. /Bruce