From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01on0064.outbound.protection.outlook.com [104.47.2.64]) by dpdk.org (Postfix) with ESMTP id 0BAAC1BB42 for ; Wed, 11 Apr 2018 19:18:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=o6P/0GY4ktuzy2rHlbuU+exdjAVeJY4/HFRwIhMsm7c=; b=VbtXVObYdEwtf5zPsVzpK/k86wxefxcRe/3y4lIB/e2ks92OtIdDF/CHoxEvoj6KeEuLL0Ddyob1ul7pDyCkTojIDQcFauu6+TuAPn8jEamh61hxJU6iATuC74+pWfM04xBb6ahcW9Ftg+OvurpbQX21Q+gK+36N0VW/sP0aato= Received: from yongseok-MBP.local (209.116.155.178) by AM5PR0501MB2036.eurprd05.prod.outlook.com (2603:10a6:203:1a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.675.9; Wed, 11 Apr 2018 17:18:51 +0000 Date: Wed, 11 Apr 2018 10:18:35 -0700 From: Yongseok Koh To: Andrew Rybchenko Cc: "Ananyev, Konstantin" , Olivier Matz , "Lu, Wenzhuo" , "Wu, Jingjing" , Adrien Mazarguil , =?iso-8859-1?Q?N=E9lio?= Laranjeiro , "dev@dpdk.org" Message-ID: <20180411171834.GB27791@yongseok-MBP.local> References: <20180402185008.13073-1-yskoh@mellanox.com> <20180402185008.13073-2-yskoh@mellanox.com> <20180403082615.etnr33cuyey7i3u3@platinum> <20180404001205.GB1867@yongseok-MBP.local> <20180409160434.kmw4iyztemrkzmtc@platinum> <20180410015902.GA20627@yongseok-MBP.local> <2601191342CEEE43887BDE71AB977258AE91344A@IRSMSX102.ger.corp.intel.com> <20180411053302.GA26252@yongseok-MBP.local> <2601191342CEEE43887BDE71AB977258AE913944@IRSMSX102.ger.corp.intel.com> <44a32fef-9087-f0c1-b28f-47b2d6a06995@solarflare.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <44a32fef-9087-f0c1-b28f-47b2d6a06995@solarflare.com> User-Agent: Mutt/1.9.3 (2018-01-21) X-Originating-IP: [209.116.155.178] X-ClientProxiedBy: BN6PR11CA0043.namprd11.prod.outlook.com (2603:10b6:404:4b::29) To AM5PR0501MB2036.eurprd05.prod.outlook.com (2603:10a6:203:1a::22) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(48565401081)(2017052603328)(7153060)(7193020); SRVR:AM5PR0501MB2036; X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2036; 3:9KSO3HdWskLjumXHXbjrodU/GMaosfHp07HjHEBTXNpq8fENiBEFXJ5oAKxEExPcSX0+QWKe/v/wy9nEPBfFCm1uZmKo8dMywSjtDkOKyB3m4l1wCGH3qUOxKUqbFt1XbQ5uBgw7o4NX6gTWeJlv1/+eLnbhiq6myc8XZacboNNCJZ/QLQa9nCRpFhn309B1DLTByjAbru7ZvSxt/GqmPFAzuZNi/xKhLtLWbEpTc9Y338NxNkeEGQQCG5xfI1Mz; 25:N1v/XwIgRAqBw9P5pcjDAWe7HdYM2a5B3iz8REuLd7J0pceHeS2wVBicpyeETF8uyl1QHwBHOoqm2r2uw0qJXiTcLj8N4n6GaY8ZdodQScfSt3xLsm8NJpqigwNhuHpBMUUddoCgo8TAqj95jQ8fRhWXuA5Nxh3pDGdRpfe5JT/EC544XI7LTsf+mCVoLekHd498QDwwcZY6Z2JZITBG6/KSPzYgyn0bK1+djMtfK+KnBcdF61jmEpDHwgUToCugETepiMpdpy6+nGj6MeMnnIUsFISjJ7B81XE5QDyi2XgSiPQjmyqyx7O6ch3bIa5qNjKRKJLGCSoKkYKBIinxww==; 31:cgUMdU4krFHbyEA+hzcsBnneiurWC4urDlM/7CIvJT/jKs75ep/pgosQwweVbJUr8DdaXz/9As7+LXKbtepvI4utrcP7keG4kdP9+UComHuqfC0IG9uI/AMrCVoqPkbhb/Ad1JWpscPdLjwLtw/TABBioFy4Qt4hDKal3hKJGPqHcnYaty2HT0p20R2kIeYDtuBzamgCsi+CT0UbSc1QekYnNGarxuN3CQZgmOBqgGA= X-MS-TrafficTypeDiagnostic: AM5PR0501MB2036: Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2036; 20:+AygdeYd9/uJUik385xRVWyw8ey+eSJAar0JFqKrDb8Vj9I7kRpuiJCDHLT3M7F13xIZNK5MRZoARPBYF1WFAJD1YJrG/pre7LCUCHUwacGLY/OhuvilARKV6fp0j3e32oY2UzUySnAhmquzkbRiMghJtRZuZMX6/QdLlPZRUjwTiKLfFxXVCBxLkALlvXY22FdNXEWI8sgU1Imiz5WM5yqGBdHDq38cdgvs5vIyohx4qvIfWzmQW7PguF3TCPepXGfjA0IVYGjK40IdPB0E0rbWaSyiU86V2BzaWbm76joWUcSR5+qu8+V4N1aRhOVehMkAiK7V/RQouAzM1ML+Rj8Odjo5KZ+Yim/T2GfWQ8wxHeoahrpKmmIOjwjNF6IoNLl7GhFXrZfXyDwy+vZ0f6LLZoJ7FSSypeT7VEealoPUQNeYTFqKR6dWG2C8wqmcpja235QMtLOwaiEn5gep0XSnIEHCIsL3C+kqEPNj25DVIXLKE7PhQP8TkwAHLkGT; 4:SkJHlMYuTk6NMGzS36ibHVJTXNwd1ZusZtZtaeUmuw3jISXBwQa87CP++lxTEmvlJ8NZ1PqybwA+2VvK6WYA7tq31TiofOt06bVTGGcF2dCMMWaIBjSAy8kNY90Gg9pg0OZzb3jGEhpMqOWJEXwfvoJLGqHiwi8p5oRfmOmhK4MJ5czJl8Zku+mkxHIYX6Lfs7gZesJmKbvMBlqKtAFxDDMywwM2mG6fVH+ZgPGhD/GUPOwNJITH5SpDqXyAFfkwHkFCERqASTvz3BQuHZvmwW50BNImPccp9YUMwy6emHzqBtd9UZKbUBKhNYIBI32w X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(17755550239193); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(93006095)(93001095)(3002001)(3231221)(944501327)(52105095)(6055026)(6041310)(20161123558120)(20161123560045)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(6072148)(201708071742011); SRVR:AM5PR0501MB2036; BCL:0; PCL:0; RULEID:; SRVR:AM5PR0501MB2036; X-Forefront-PRVS: 0639027A9E X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(346002)(376002)(39380400002)(396003)(366004)(39860400002)(199004)(189003)(16526019)(105586002)(33896004)(76176011)(7696005)(52116002)(6506007)(386003)(59450400001)(55016002)(9686003)(106356001)(33656002)(1076002)(98436002)(6246003)(186003)(53936002)(26005)(66066001)(23726003)(53546011)(50466002)(3846002)(86362001)(305945005)(5890100001)(25786009)(486006)(7736002)(4326008)(5660300001)(6916009)(6666003)(478600001)(47776003)(229853002)(6116002)(81156014)(93886005)(476003)(956004)(81166006)(68736007)(446003)(316002)(8676002)(97736004)(54906003)(11346002)(2906002)(16586007)(8936002)(58126008)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:AM5PR0501MB2036; H:yongseok-MBP.local; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; AM5PR0501MB2036; 23:fgdGyK3JBV80WJ61ape1P4oIYmAX2fznZf1VizU?= =?us-ascii?Q?70yCOwE6hjdkoYjhCWfg/tj7az6gfX5qWosuHBgWZKsHzLZAyxWQJ7JZpDIZ?= =?us-ascii?Q?vp4JEpEaCYiUbuE0msurlUufyzMOILBPFhxb4+9sQBjZLRACe79dcrbpgszp?= =?us-ascii?Q?lSFtP4wrJOD4X16waxQgXZElHEvZ3FVvJTLk+mao1Ww6xeecbEElNjFZtcMo?= =?us-ascii?Q?Yd+nSqoCqrA0yDdH69CryLIHm7bgzvSsi+XyxOHg5M8zHSBG2WJUN8dNNEXP?= =?us-ascii?Q?dTq+lUibFq7lTU2dcoXT2ph/W2xUiGQo7yKprhN5jGHTrzIr2LAnyBt95g65?= =?us-ascii?Q?jwfF7IQGnY2hSAZruljiIj7+dTIYiUw2CiwC5BAapIpfnQVPivZ6SsUgkTEg?= =?us-ascii?Q?W0nbtoO7Y5u+6R4k6fofBI9dxH0QWymfe27vP6qbW1nFyAnsss342X64RnIm?= =?us-ascii?Q?DwyfsGnEVJ1pwqD3v8NTMhLVPZ3w5ecx8ma0ybsrPpGwvbK4qGDtCI/kXF1p?= =?us-ascii?Q?bk9u+W74f6xlgjCsv8yetrzvWmBhmoy+TJHJykClENU1Cg9H5RTP/RybcJUT?= =?us-ascii?Q?QLzDphQSbuSwfrA2xCxN4Fy2Fi0cllydhRmzCVb8WL+SGsyl4zY98/wg2Jj4?= =?us-ascii?Q?DZJk6wU2qKkICjA390qJOmY6GAoICmGgYeOsJ7vvB/fLaHC5PpG7wH9eLWml?= =?us-ascii?Q?Z+sNoLr58WTA5tm2sNOxAXXDdyfRTXZVeRv1zX2z50lloFi19qVeG/lZk4PE?= =?us-ascii?Q?12XAmcdPH6EnC8eDAfEHQA3lnhtijLAIcOd82FbfHzbkdRDG5NYQ57blihgP?= =?us-ascii?Q?75L6ofQMfJSI324uVr9qkjFfxxBV67uDvBIwUSsKnpZl38VofyZWGarKcUyR?= =?us-ascii?Q?WhnG8+KJoOt/AgDmynd0MMqp3xxSc4Y8MEti+XNHMLmXrzo6FGG3XKtq2aou?= =?us-ascii?Q?G6uG09u7XGq0gCiZAl0c8UCTRznkkPX6plnPHssq1cX5QKxJlN7sdap37F9R?= =?us-ascii?Q?RvwSnfuzMfIvatzAGcXkHxOeHHWSo092ox6L94ommfUBfHdEw0iJEMxK0h9y?= =?us-ascii?Q?UeP6L1gGziVXmiTpC1r4hp8AeMknhT49uQZDuNIMf7qrvXfYmHyGnZLKJRUv?= =?us-ascii?Q?03vgWoW0mZWw5xNx1cvUjAHzhPDk14UpY6G78eC9Znvg/ISr0HZHk0V1NfsG?= =?us-ascii?Q?tYug8Ieki2n7S1N0lIJRas1wJ69OJVWmCIIswMDMPg8csxp1p9SCh4Zwlr0r?= =?us-ascii?Q?ACbZzYiHB/9BaGVsaM1cMfvXxCB2bzezG5GI7D2a/CvMK/Ijut0u4kfYqe8K?= =?us-ascii?Q?opv9O9aXtzFh0+wZvwIA/Pu0KKp/BOyuTPhsSKRDbvoa2RZEkStC+1Bpq6yX?= =?us-ascii?Q?9xIwgWM1+sDgzu97u3dm7SQs2qVqz/RVpwxGvgK0D3/RqntQ8?= X-Microsoft-Antispam-Message-Info: N5GNgkI8A1HVyO5pD2kQ/k1h6F4EXVHEY4GdQOP3Oudo6BPCWryv4vEL15y3Fj8B/dTAqe1zJRfVieLCjM9Ld0TLAQYDrPsrWwcccOG507IbjBvXW50PdwbACfp3euj3pTOE+rodld0fqFQflo/UWiqAW6JhkP+bQ6sxupC0u5KkoZwdyZmSSBoWiHu+RS0l X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2036; 6:ohhn9qUAbiR9bSgl61pxBnae6eYKoA/k4rG7IPNV0VMATTsa4Wja31ndGw2ATcq1xsGxJk7d0JMeN3QFBX28+/IAr70p6B54wRVNymBe13txxBUc5dmbiBgnI2WjYojCFkrhcJpDzGJHzto+NMPoUNnYsqgUHhgQUiGGBlxoEIkBOBs9NsQl2WXORdzLiCKS4okX6qSq54GpkmNp835UTr+seGuQnGOttiWGhM4RjB6quzMeLMNT6lRPWCnOezuodOlCm06PfDT4KmDx7iBhAWA+td5yBCWesWmYcLnj+AY1Ms3RVI4HyGuiqTpMWSd13ZifKebQgTgqyTsLOQQzH0CmankUBWNRgZeeVQAk93MEKuEIRnZp2UyNNrLHNOMg0TkIXKMB2MUgY/0ar6azYqsxcGYpW0Twl0zfD1bcPeOAaYEz6ll3ghW45o8JsO7qDR276jS+r0Ycwtwy7P8Tbw==; 5:baUWECe5tXTXMHZ9THriZ5CuR70NxRPBuliTZBD1VKilZiQEZ7idA5CMB2sfZz4VcDVIPQdG89t5laI9qEM2IDba+E/tWUIn+HymfOpa4BUA7PV4O8QeH2errLlxvBFAVgMdJ/eIqtaicpHhVH8L7fE4ALZ4Y+ETESCfq3bvIwQ=; 24:CkniZzSaFZjfWE0rWPZ3Kr8nP1XpaUDebJUktndyfjKlxMpo3l5dGB2eqcOtYcY5uar54zi5A+hF1+dFORShS3JMjrE9Z846HgEZaD6khgk= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2036; 7:8vPz6oaJlq7tcMKP+n4mqNmSYyajzrOOdP6ufXgKVDJlJciIDDQfKhvQjDqxnt3YMarZXOTMmKgGSxRnKRrOaH4UufS9tD743linRc0yUtxfJUBit50KvVdQy0K7kFgLac22yfqRuCZstFwy6TBgh6LV/kWGe2PuJGNwdXujEEl01k5XstmowaRRlrHu/RsXwjgWHyAeWjeFVWe5DDN7MLOJzzW+IP7eFGQGfMJutziLcMQH/+1/InuJNoPxaL17 X-MS-Office365-Filtering-Correlation-Id: eabcc041-5bac-400b-7c40-08d59fd04ccf X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Apr 2018 17:18:51.7199 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: eabcc041-5bac-400b-7c40-08d59fd04ccf X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0501MB2036 Subject: Re: [dpdk-dev] [PATCH v2 1/6] mbuf: add buffer offset field for flexible indirection X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Apr 2018 17:18:55 -0000 On Wed, Apr 11, 2018 at 05:02:50PM +0300, Andrew Rybchenko wrote: > On 04/11/2018 02:39 PM, Ananyev, Konstantin wrote: > > Hi Yongseok, > > > > > > > On Mon, Apr 09, 2018 at 06:04:34PM +0200, Olivier Matz wrote: > > > > > > Hi Yongseok, > > > > > > > > > > > > On Tue, Apr 03, 2018 at 05:12:06PM -0700, Yongseok Koh wrote: > > > > > > > On Tue, Apr 03, 2018 at 10:26:15AM +0200, Olivier Matz wrote: > > > > > > > > Hi, > > > > > > > > > > > > > > > > On Mon, Apr 02, 2018 at 11:50:03AM -0700, Yongseok Koh wrote: > > > > > > > > > When attaching a mbuf, indirect mbuf has to point to start of buffer of > > > > > > > > > direct mbuf. By adding buf_off field to rte_mbuf, this becomes more > > > > > > > > > flexible. Indirect mbuf can point to any part of direct mbuf by calling > > > > > > > > > rte_pktmbuf_attach_at(). > > > > > > > > > > > > > > > > > > Possible use-cases could be: > > > > > > > > > - If a packet has multiple layers of encapsulation, multiple indirect > > > > > > > > > buffers can reference different layers of the encapsulated packet. > > > > > > > > > - A large direct mbuf can even contain multiple packets in series and > > > > > > > > > each packet can be referenced by multiple mbuf indirections. > > > > > > > > > > > > > > > > > > Signed-off-by: Yongseok Koh > > > > > > > > I think the current API is already able to do what you want. > > > > > > > > > > > > > > > > 1/ Here is a mbuf m with its data > > > > > > > > > > > > > > > > off > > > > > > > > <--> > > > > > > > > len > > > > > > > > +----+ <----------> > > > > > > > > | | > > > > > > > > +-|----v----------------------+ > > > > > > > > | | -----------------------| > > > > > > > > m | buf | XXXXXXXXXXX || > > > > > > > > | -----------------------| > > > > > > > > +-----------------------------+ > > > > > > > > > > > > > > > > > > > > > > > > 2/ clone m: > > > > > > > > > > > > > > > > c = rte_pktmbuf_alloc(pool); > > > > > > > > rte_pktmbuf_attach(c, m); > > > > > > > > > > > > > > > > Note that c has its own offset and length fields. > > > > > > > > > > > > > > > > > > > > > > > > off > > > > > > > > <--> > > > > > > > > len > > > > > > > > +----+ <----------> > > > > > > > > | | > > > > > > > > +-|----v----------------------+ > > > > > > > > | | -----------------------| > > > > > > > > m | buf | XXXXXXXXXXX || > > > > > > > > | -----------------------| > > > > > > > > +------^----------------------+ > > > > > > > > | > > > > > > > > +----+ > > > > > > > > indirect | > > > > > > > > +-|---------------------------+ > > > > > > > > | | -----------------------| > > > > > > > > c | buf | || > > > > > > > > | -----------------------| > > > > > > > > +-----------------------------+ > > > > > > > > > > > > > > > > off len > > > > > > > > <--><----------> > > > > > > > > > > > > > > > > > > > > > > > > 3/ remove some data from c without changing m > > > > > > > > > > > > > > > > rte_pktmbuf_adj(c, 10) // at head > > > > > > > > rte_pktmbuf_trim(c, 10) // at tail > > > > > > > > > > > > > > > > > > > > > > > > Please let me know if it fits your needs. > > > > > > > No, it doesn't. > > > > > > > > > > > > > > Trimming head and tail with the current APIs removes data and make the space > > > > > > > available. Adjusting packet head means giving more headroom, not shifting the > > > > > > > buffer itself. If m has two indirect mbufs (c1 and c2) and those are pointing to > > > > > > > difference offsets in m, > > > > > > > > > > > > > > rte_pktmbuf_adj(c1, 10); > > > > > > > rte_pktmbuf_adj(c2, 20); > > > > > > > > > > > > > > then the owner of c2 regard the first (off+20)B as available headroom. If it > > > > > > > wants to attach outer header, it will overwrite the headroom even though the > > > > > > > owner of c1 is still accessing it. Instead, another mbuf (h1) for the outer > > > > > > > header should be linked by h1->next = c2. > > > > > > Yes, after these operations c1, c2 and m should become read-only. So, to > > > > > > prepend headers, another mbuf has to be inserted before as you suggest. It > > > > > > is possible to wrap this in a function rte_pktmbuf_clone_area(m, offset, > > > > > > length) that will: > > > > > > - alloc and attach indirect mbuf for each segment of m that is > > > > > > in the range [offset : length+offset]. > > > > > > - prepend an empty and writable mbuf for the headers > > > > > > > > > > > > > If c1 and c2 are attached with shifting buffer address by adjusting buf_off, > > > > > > > which actually shrink the headroom, this case can be properly handled. > > > > > > What do you mean by properly handled? > > > > > > > > > > > > Yes, prepending data or adding data in the indirect mbuf won't override > > > > > > the direct mbuf. But prepending data or adding data in the direct mbuf m > > > > > > won't be protected. > > > > > > > > > > > > From an application point of view, indirect mbufs, or direct mbufs that > > > > > > have refcnt != 1, should be both considered as read-only because they > > > > > > may share their data. How an application can know if the data is shared > > > > > > or not? > > > > > > > > > > > > Maybe we need a flag to differentiate mbufs that are read-only > > > > > > (something like SHARED_DATA, or simply READONLY). In your case, if my > > > > > > understanding is correct, you want to have indirect mbufs with RW data. > > > > > Agree that indirect mbuf must be treated as read-only, Then the current code is > > > > > enough to handle that use-case. > > > > > > > > > > > > And another use-case (this is my actual use-case) is to make a large mbuf have > > > > > > > multiple packets in series. AFAIK, this will also be helpful for some FPGA NICs > > > > > > > because it transfers multiple packets to a single large buffer to reduce PCIe > > > > > > > overhead for small packet traffic like the Multi-Packet Rx of mlx5 does. > > > > > > > Otherwise, packets should be memcpy'd to regular mbufs one by one instead of > > > > > > > indirect referencing. > > > > But just to make HW to RX multiple packets into one mbuf, > > > > data_off inside indirect mbuf should be enough, correct? > > > Right. Current max buffer len of mbuf is 64kB (16bits) but it is enough for mlx5 > > > to reach to 100Gbps with 64B traffic (149Mpps). I made mlx5 HW put 16 packets in > > > a buffer. So, it needs ~32kB buffer. Having more bits in length fields would be > > > better but 16-bit is good enough to overcome the PCIe Gen3 bottleneck in order > > > to saturate the network link. > > There were few complains that 64KB max is a limitation for some use-cases. > > I am not against increasing it, but I don't think we have free space on first cache-line for that > > without another big rework of mbuf layout. > > Considering that we need to increase size for buf_len, data_off, data_len, and probably priv_size too. > > > > > > As I understand, what you'd like to achieve with this new field - > > > > ability to manipulate packet boundaries after RX, probably at upper layer. > > > > As Olivier pointed above, that doesn't sound as safe approach - as you have multiple > > > > indirect mbufs trying to modify same direct buffer. > > > I agree that there's an implication that indirect mbuf or mbuf having refcnt > 1 > > > is read-only. What that means, all the entities which own such mbufs have to be > > > aware of that and keep the principle as DPDK can't enforce the rule and there > > > can't be such sanity check. In this sense, HW doesn't violate it because the > > > direct mbuf is injected to HW before indirection. When packets are written by > > > HW, PMD attaches indirect mbufs to the direct mbuf and deliver those to > > > application layer with freeing the original direct mbuf (decrement refcnt by 1). > > > So, HW doesn't touch the direct buffer once it reaches to upper layer. > > Yes, I understand that. But as I can see you introduced functions to adjust head and tail, > > which implies that it should be possible by some entity (upper layer?) to manipulate these > > indirect mbufs. > > And we don't know how exactly it will be done. > > > > > The direct buffer will be freed and get available for reuse when all the attached > > > indirect mbufs are freed. > > > > > > > Though if you really need to do that, why it can be achieved by updating buf_len and priv_size > > > > Fields for indirect mbufs, straight after attach()? > > > Good point. > > > Actually that was my draft (Mellanox internal) version of this patch :-) But I > > > had to consider a case where priv_size is really given by user. Even though it > > > is less likely, but if original priv_size is quite big, it can't cover entire > > > buf_len. For this, I had to increase priv_size to 32-bit but adding another > > > 16bit field (buf_off) looked more plausible. > > As I remember, we can't have mbufs bigger then 64K, > > so priv_size + buf_len should be always less than 64K, correct? > > It sounds like it is suggested to use/customize priv_size to limit indirect > mbuf range in the direct one. It does not work from the box since priv_size is > used to find out direct mbuf by indirect (see rte_mbuf_from_indirect()). ?? That's exactly why he suggested to use priv_size... Thanks, Yongseok