From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20070.outbound.protection.outlook.com [40.107.2.70]) by dpdk.org (Postfix) with ESMTP id F04E31B781 for ; Wed, 11 Apr 2018 07:33:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=5n+eO4wUI8w/mimLp1wAxI58hQ2bA6lmyFq/OGjTrv8=; b=N3j6jn96XgEIod41hnth6EUFoO3WDne5spvie1Mx4vo7PDfhCCVVDDxEZuYvJcmJu8+wNdV6FYrf2fKc5S9FFC0Pbtd9YDpAw+woXULuzSZZSxSs/1RNBf2GpSwExmZvmVSVUdbKsH3cmu/dKNnLGfUtIlPthUj2W6mMzBAA0Po= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; Received: from yongseok-MBP.local (73.222.116.174) by AM5PR0501MB2033.eurprd05.prod.outlook.com (2603:10a6:203:1a::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.675.10; Wed, 11 Apr 2018 05:33:19 +0000 Date: Tue, 10 Apr 2018 22:33:04 -0700 From: Yongseok Koh To: "Ananyev, Konstantin" Cc: Olivier Matz , "Lu, Wenzhuo" , "Wu, Jingjing" , Adrien Mazarguil , =?iso-8859-1?Q?N=E9lio?= Laranjeiro , "dev@dpdk.org" Message-ID: <20180411053302.GA26252@yongseok-MBP.local> References: <20180310012532.15809-1-yskoh@mellanox.com> <20180402185008.13073-1-yskoh@mellanox.com> <20180402185008.13073-2-yskoh@mellanox.com> <20180403082615.etnr33cuyey7i3u3@platinum> <20180404001205.GB1867@yongseok-MBP.local> <20180409160434.kmw4iyztemrkzmtc@platinum> <20180410015902.GA20627@yongseok-MBP.local> <2601191342CEEE43887BDE71AB977258AE91344A@IRSMSX102.ger.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2601191342CEEE43887BDE71AB977258AE91344A@IRSMSX102.ger.corp.intel.com> User-Agent: Mutt/1.9.3 (2018-01-21) X-Originating-IP: [73.222.116.174] X-ClientProxiedBy: CO2PR04CA0002.namprd04.prod.outlook.com (2603:10b6:102:1::12) To AM5PR0501MB2033.eurprd05.prod.outlook.com (2603:10a6:203:1a::19) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(48565401081)(2017052603328)(7153060)(7193020); SRVR:AM5PR0501MB2033; X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2033; 3:jodm9PCVPSt8b4wFI/f0i2pwJA1+qUn2lAGDhuzxhg9Y+Ji0AED+y85vi5RwvXptT2qeWGYlkmmmyAIDdo1G99aURWXpD6ccyybXRrWY+qCmBQ6XJkt0IBRVi6Le+ux0Zple+Z1s2myJM/zsGjgtervYK0h5yBlZURYRyKfRq6azLlp5Q8MeEYwZehbEYZHmOTD+kdR+b8NUajenp/DyM5IdxBx6RNmXZalqsAK2LquC9yKCtCF8TQ2Nq65JH10I; 25:y1V8VhZ8kCrfP+yvSwc91kUazrv/Qw5CfxUA5B5oJmmGSizRMP6VONR0tRXiVFINIVRbIM4LAQXUW6XBh5O82lug1GOD2Uqo6Ds31M5b/E3FHbIpeuRx89exFb5LLkGh8PyiiWabK2SYX3uk3QObMwcsCUIh44yd0m1n583EgFr1HSK3fqIhDQ2+w5oz9RtUGWe9fLDggp9xPm6h+E/COhoVrUGWsU0c7Az9HRYqKdsN9LapgPlZHsVnzDXJ8IAWYDaGgqNdIbXij0IqErrAiUML9a1DtvG0MLq/kJuniRDxyov/PIbYQCbGBiKcHDzhjtO9TBgWl8LxaOcom+qa6A==; 31:xxLeeXweCbRwZ2UiEaMZ2mdYHM+6RK23M7N0jhEMveznFqXjCwdf+LaPh1XatSVxP6w1R3N/YGzMU1FWp+r5JWzlYy96oGkat8FlmnZgQyzgqPDLxKoIT5wiUjpx+m7e6ALzaFARKCdKEA2R9sN1e+xVAtgT34AXJ1UfWOzM59iiNbX2xRhPRfGzz69NSe57eOtJaRHA+Oy/YImTEmipuSaUPL3PnKT8SWkXU0jFyvc= X-MS-TrafficTypeDiagnostic: AM5PR0501MB2033: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2033; 20:GXnxAmsBxJIrQMdYG6kO20mi0cyq//yxdB+qoPCZ5Up+7em3sjEFPaJ7ixLxH+dTK5wpOsYe7kIHVvexPS2adO+3wUjNlqytCUvT9nKuABA5wjOQj6eW3+NGGS/t7L/0pzJRkabSgqceAb5imlsZTQ4Ke0ch9nE/1PsdAs1fl2w70jY1/79GiXKTFHS5MwzxZYZC1MPl4dOYOqtqWxMlKnRfRU/cZykQRmaay+AiohQF3EUg3JNTdt+bskcZ1XAiw2jOQ713Ynv3swpUwLaJcV1wo/UkeviDjaFdWBj1s1lkUizMPyr9UFv8yv6yP34ZWVc+jA18KK2V0eK7lzra411d4jYO/LViJ96oqq8Cp5l5zSXVH9jyQaaZmkOx58LKxvmBY25HVFM8tDRRY77Kh9v0naCJc/9+DXOC0pv2J45aEcgUQ42JBdr1TN6U/sfGUzJCN8Wz8P1paR9xU2qPCirb2LbLzOpTSlkJAMFmpyvTvDq08WzrxXYj836/dak7 X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(189930954265078)(15185016700835)(45079756050767)(228905959029699)(17755550239193); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(93006095)(93001095)(3231221)(944501327)(52105095)(3002001)(10201501046)(6055026)(6041310)(20161123560045)(20161123562045)(20161123558120)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011); SRVR:AM5PR0501MB2033; BCL:0; PCL:0; RULEID:; SRVR:AM5PR0501MB2033; X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2033; 4:aJiEuhcq+lIlFDnAoFkbx8Ai8N7YeFlBIx4h7kewdtS6n9x7EialdG0wQm1WNCX4riFNn7tOCoDAfVqqugLKqmzVYA4OYM1p2iK2JKgjM0H6vKfa5V1WRxxrtqO0CGKs+KmKfcyTI67aa/9/k88ERA2Fg5piheUxNQ+pVHQ2n3AxpmFx21F9FplCUhM/NxjmuT6WbRWMIwtC6elpfgA0Ry1oZl9S3lTfXaZUiJdEXIeR4fpOqX+Y2tNA4OZxtdM8NJJhmMIL59pTSSrub2OK4WskPfgnqW5cvcCm+cqsjmzPvPpfO8OdRVm7nRl1Y/tGkV/XcaD/s8DYA+HSc/M0FTK6e5yqDcuqktbSiyEdUd4z8qtCA+jzKWk3+cH6ecGuqxAV2lD3U5fR25PrbDfKKq/W0rcZkyayXnZBH2aj6olT2sbWHuT8wWqzwwoYD7dRJ6BHUSu+hWWRiz3l0W+4Yw== X-Forefront-PRVS: 0639027A9E X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(39380400002)(376002)(39860400002)(366004)(346002)(396003)(13464003)(189003)(199004)(6116002)(229853002)(5890100001)(6666003)(6916009)(58126008)(16586007)(25786009)(316002)(45080400002)(4326008)(54906003)(86362001)(93886005)(9686003)(8676002)(81156014)(53936002)(81166006)(68736007)(6306002)(486006)(33656002)(50466002)(6246003)(98436002)(47776003)(66066001)(8936002)(55016002)(7696005)(52116002)(106356001)(76176011)(33896004)(59450400001)(53546011)(6506007)(386003)(966005)(305945005)(7736002)(105586002)(478600001)(26005)(5660300001)(3846002)(23726003)(1076002)(476003)(11346002)(956004)(446003)(97736004)(186003)(16526019)(2906002)(18370500001)(19627235001); DIR:OUT; SFP:1101; SCL:1; SRVR:AM5PR0501MB2033; H:yongseok-MBP.local; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; AM5PR0501MB2033; 23:KTzl9dBtr8xVN65YwG1ns1k+4PEaVAvtrzvAFSA?= =?us-ascii?Q?/MHDu/mboSrXhSqECdSybFWo9SqHgm3PbQbGuoWMApPSyzj5jXox1hOFpNdF?= =?us-ascii?Q?OntdSHR4pEiM9O45cq4mlGFrAeiGlOUOsZeaR8dK4q7rjZLbtCVaAI2S8pcM?= =?us-ascii?Q?sX8vsDIUGCVmjALjbm0ybH3EV857VArSpNkZ0zvTUa6NSMk8nWyAx3I28Hrx?= =?us-ascii?Q?lg3LEUWchrjOBsPxyJevQpU/RiNqKD3teOkcqqWqrfGknmoatqNm8mrg4KFg?= =?us-ascii?Q?q1MXMcA4pYmFKv/LjiD59lsosTKpJiHC6dBv7EMM0aR7T2uNRn84vvx5L+L6?= =?us-ascii?Q?dt350dzVolAv7WdJfA6M2mBYYQ3UMJ0O67ApGFRFYeQQBgCi0NCURVpNS5cl?= =?us-ascii?Q?5cFILq0wsWOevxde06vKGqZKk2j+FaeolI/0BVLMOK3UA/sGPRmED3aQm+hK?= =?us-ascii?Q?n4B/R6xJ+sJ5sg3jEBl9ILwZ6OT79Lk+06ebmImawVA6G7oZSi1b2k2pqoE7?= =?us-ascii?Q?KhAIyfwYqVv73gfhbvgOg/6b+R9WpkWduJl+t0QUzCRdJBRKoXgT4JUcYiVN?= =?us-ascii?Q?JUo0bfpRWZlWWXtxBTuVt9d+Ee4aTLqzPuZyVGQ0OBgjORdbLz3UUqpQ5iKL?= =?us-ascii?Q?hdsfx0Z6RVeJRL5yv5DC2L6h/I5Qxq14gMhYnJS/6pqx+wctBHL5QZaQI3oY?= =?us-ascii?Q?6UDFCfCj4pCVC8hUom8u98ID5yq2c+2NpwrEiiiVQH+PzZZjUBrqRhtMgnzX?= =?us-ascii?Q?nDi/LjiThnkHALtcMOLS98ULGEcjTaokQGyOii/FdXjnvXiuwbjIVEiXYIj7?= =?us-ascii?Q?ymk28h0okATW9Pqdz58lzW+fsWsZ8tZIGBGvm/B1h/DSZdvas3cee9diKrFx?= =?us-ascii?Q?b3G8gxmudfUP13/oJR/WJcUpM9Td93eRcRanEL4hDB0KuGyOa2Xbc3/teeYG?= =?us-ascii?Q?wMTIhKt75EV1pjF+yjsxEhUqcm0UHZA94wUzHGameCAiuqhC3awrHqLbPt7q?= =?us-ascii?Q?Bluh+NMDdjmtp3OuJd48tweJYqPYXwtEmk6+yJ7lmnvBoEwoj+fflKeSwtlH?= =?us-ascii?Q?yAMexxoJ6Tb1tJ/3oBrkUw6H6FbWt/od1CJqO6HMr6w2wkIdsvZCv4+iFbxe?= =?us-ascii?Q?tnVISM9dJCVvFsK8Ebkf6JY1dWQJ76BEyrsrcYbgGgo34MVIyiMXLxlATYRU?= =?us-ascii?Q?e1+re2I79nEDNAvywKlFxlSqQJ0rGC7XC53lqNsf2RPeXkm3mE48CEbaXvbO?= =?us-ascii?Q?jpSsxbzFqJWLk7aOlfbt47mDX4DgOyY5D5Lnl63dQWTD1tOtdG8SRQYZKGrD?= =?us-ascii?Q?xKQyNZNUW6EZzJ6SPU3RDolhYPHfdYLMcUMQKJONEVAwFFH7YIEJIpfWJMx0?= =?us-ascii?Q?RuptoeX2RThSQzG4l+zNwkGqaL0H5yBeqZLfp/o96qAFMEEFKVlEPrdVZkLA?= =?us-ascii?Q?ReRCxiorIDjEqtz1bUqb4gdvZSRP6UXDcx3uq5hOfHtD2h3eo4XJ7eQ5JUAp?= =?us-ascii?Q?o3VG5ln/ODzQLz98fX9IpnvRwZwGASpBON48=3D?= X-Microsoft-Antispam-Message-Info: t0fmZgilYrB6Zjyur6FY4ShnA1q6w6THZop+WBhHwoibpWPq3sAQEJMt5elckaHQIpt3bSviyIXfOHUJDpSEzYIvMTyq1H+BszedH6IYC3Ylxcetv0cnzhHFIQepE82Nink5SbK5haO/bNXkR3iWL3vjGnQcjxkD+9u0/n/y4/J+Agv9vjW5ehaD9vyB90n3 X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2033; 6:gFD3IJSx2CYupbEzDDyQA55N2lMg94Z/Njia0jFkpuUKzbUyqhdSipXmZEkQ6GhG13f2fnJPkt1KIfvaegZXvldoIbuv3iKIGEv/1n+Z8w6Ga6tNZsdKO77to+j4tJTJwtT3wPhs2CkT1Ee4D/9/YWOnU5VodECJ9TihjVZqYKFZCRtyojCKShfxe3eJ9wjZNxOCZsihG6bPI6mLaOqjvvK8+ICgfrlfHdD6qJMFjGpRLllUyZhh89RJt0WcuOZEVvj/9lqnamvbwbSje+EBQAa02/y4eM5VYt7rKpmKqQbN454eQTSz0CuZgtRWPiMtEDe8GmYO7MYz1H3suk09P5ioZvU2Ga4A9rtAC+pf6BVsSnsqQInb/k692LjnexVa0aajkzvT23t0bceHQxJkwgEpk/XLJGhAIH5uBIFwAaJW14Im2GEHr7UZwvYxLoVaU04YUEjIqjcAfjuOuPnqXw==; 5:VSH05uj7ceX8/3URQenPlogpXxP2Twx9uzN2Yxw80bDMIUpZL57DyViKqinoKKuxFvGDTMo8A87Hl9jChlYYe6uPwZmTo1/YNPy9ciU4UUtQkFtc+/H8AoiEf89f04n29VainOYNABcqxBKsEgPUZPZT8/J3Getpgl4Y2wGcN6M=; 24:YI77AygaNOLN8MKbj23tFghUO8Z7clA4xDNYbyMIRWtZD8yIzFM9MXFrhf4SIEmQXIAaTHJdIKSWPiwYcAn65XX8CeCPyHhgRYowtHV6x18= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2033; 7:LcLC7kJpwmtv6WEljRaHEcVNzMmmdkncRZ0jiKaGqyHOuE437IA+yHQN5Obm94zCCjs7B7K4OKw4b+vBBhPB/IXCdr6pcReC2iQ5863PfENyVn6TsEStQ/5a8edScJjPX32uV/AITfgenm9Sabml8DICV+vArmsPnTAY0V0PzyBqnCsMuVCSjVy113sS1UYhVz4kpUR9YBbTLjHyocHz3IUSs7slc5WG4ehr0Zh/q1KQrd9vwQfDCaP3eDuxd3Kx X-MS-Office365-Filtering-Correlation-Id: 2b678d42-5bc1-423b-a11a-08d59f6dbd25 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Apr 2018 05:33:19.9350 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2b678d42-5bc1-423b-a11a-08d59f6dbd25 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0501MB2033 Subject: Re: [dpdk-dev] [PATCH v2 1/6] mbuf: add buffer offset field for flexible indirection X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Apr 2018 05:33:24 -0000 On Tue, Apr 10, 2018 at 05:25:31PM -0700, Ananyev, Konstantin wrote: > > > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yongseok Koh > > Sent: Tuesday, April 10, 2018 2:59 AM > > To: Olivier Matz > > Cc: Lu, Wenzhuo ; Wu, Jingjing ; adrien.mazarguil@6wind.com; > > nelio.laranjeiro@6wind.com; dev@dpdk.org > > Subject: Re: [dpdk-dev] [PATCH v2 1/6] mbuf: add buffer offset field for flexible indirection > > > > On Mon, Apr 09, 2018 at 06:04:34PM +0200, Olivier Matz wrote: > > > Hi Yongseok, > > > > > > On Tue, Apr 03, 2018 at 05:12:06PM -0700, Yongseok Koh wrote: > > > > On Tue, Apr 03, 2018 at 10:26:15AM +0200, Olivier Matz wrote: > > > > > Hi, > > > > > > > > > > On Mon, Apr 02, 2018 at 11:50:03AM -0700, Yongseok Koh wrote: > > > > > > When attaching a mbuf, indirect mbuf has to point to start of buffer of > > > > > > direct mbuf. By adding buf_off field to rte_mbuf, this becomes more > > > > > > flexible. Indirect mbuf can point to any part of direct mbuf by calling > > > > > > rte_pktmbuf_attach_at(). > > > > > > > > > > > > Possible use-cases could be: > > > > > > - If a packet has multiple layers of encapsulation, multiple indirect > > > > > > buffers can reference different layers of the encapsulated packet. > > > > > > - A large direct mbuf can even contain multiple packets in series and > > > > > > each packet can be referenced by multiple mbuf indirections. > > > > > > > > > > > > Signed-off-by: Yongseok Koh > > > > > > > > > > I think the current API is already able to do what you want. > > > > > > > > > > 1/ Here is a mbuf m with its data > > > > > > > > > > off > > > > > <--> > > > > > len > > > > > +----+ <----------> > > > > > | | > > > > > +-|----v----------------------+ > > > > > | | -----------------------| > > > > > m | buf | XXXXXXXXXXX || > > > > > | -----------------------| > > > > > +-----------------------------+ > > > > > > > > > > > > > > > 2/ clone m: > > > > > > > > > > c = rte_pktmbuf_alloc(pool); > > > > > rte_pktmbuf_attach(c, m); > > > > > > > > > > Note that c has its own offset and length fields. > > > > > > > > > > > > > > > off > > > > > <--> > > > > > len > > > > > +----+ <----------> > > > > > | | > > > > > +-|----v----------------------+ > > > > > | | -----------------------| > > > > > m | buf | XXXXXXXXXXX || > > > > > | -----------------------| > > > > > +------^----------------------+ > > > > > | > > > > > +----+ > > > > > indirect | > > > > > +-|---------------------------+ > > > > > | | -----------------------| > > > > > c | buf | || > > > > > | -----------------------| > > > > > +-----------------------------+ > > > > > > > > > > off len > > > > > <--><----------> > > > > > > > > > > > > > > > 3/ remove some data from c without changing m > > > > > > > > > > rte_pktmbuf_adj(c, 10) // at head > > > > > rte_pktmbuf_trim(c, 10) // at tail > > > > > > > > > > > > > > > Please let me know if it fits your needs. > > > > > > > > No, it doesn't. > > > > > > > > Trimming head and tail with the current APIs removes data and make the space > > > > available. Adjusting packet head means giving more headroom, not shifting the > > > > buffer itself. If m has two indirect mbufs (c1 and c2) and those are pointing to > > > > difference offsets in m, > > > > > > > > rte_pktmbuf_adj(c1, 10); > > > > rte_pktmbuf_adj(c2, 20); > > > > > > > > then the owner of c2 regard the first (off+20)B as available headroom. If it > > > > wants to attach outer header, it will overwrite the headroom even though the > > > > owner of c1 is still accessing it. Instead, another mbuf (h1) for the outer > > > > header should be linked by h1->next = c2. > > > > > > Yes, after these operations c1, c2 and m should become read-only. So, to > > > prepend headers, another mbuf has to be inserted before as you suggest. It > > > is possible to wrap this in a function rte_pktmbuf_clone_area(m, offset, > > > length) that will: > > > - alloc and attach indirect mbuf for each segment of m that is > > > in the range [offset : length+offset]. > > > - prepend an empty and writable mbuf for the headers > > > > > > > If c1 and c2 are attached with shifting buffer address by adjusting buf_off, > > > > which actually shrink the headroom, this case can be properly handled. > > > > > > What do you mean by properly handled? > > > > > > Yes, prepending data or adding data in the indirect mbuf won't override > > > the direct mbuf. But prepending data or adding data in the direct mbuf m > > > won't be protected. > > > > > > From an application point of view, indirect mbufs, or direct mbufs that > > > have refcnt != 1, should be both considered as read-only because they > > > may share their data. How an application can know if the data is shared > > > or not? > > > > > > Maybe we need a flag to differentiate mbufs that are read-only > > > (something like SHARED_DATA, or simply READONLY). In your case, if my > > > understanding is correct, you want to have indirect mbufs with RW data. > > > > Agree that indirect mbuf must be treated as read-only, Then the current code is > > enough to handle that use-case. > > > > > > And another use-case (this is my actual use-case) is to make a large mbuf have > > > > multiple packets in series. AFAIK, this will also be helpful for some FPGA NICs > > > > because it transfers multiple packets to a single large buffer to reduce PCIe > > > > overhead for small packet traffic like the Multi-Packet Rx of mlx5 does. > > > > Otherwise, packets should be memcpy'd to regular mbufs one by one instead of > > > > indirect referencing. > > But just to make HW to RX multiple packets into one mbuf, > data_off inside indirect mbuf should be enough, correct? Right. Current max buffer len of mbuf is 64kB (16bits) but it is enough for mlx5 to reach to 100Gbps with 64B traffic (149Mpps). I made mlx5 HW put 16 packets in a buffer. So, it needs ~32kB buffer. Having more bits in length fields would be better but 16-bit is good enough to overcome the PCIe Gen3 bottleneck in order to saturate the network link. > As I understand, what you'd like to achieve with this new field - > ability to manipulate packet boundaries after RX, probably at upper layer. > As Olivier pointed above, that doesn't sound as safe approach - as you have multiple > indirect mbufs trying to modify same direct buffer. I agree that there's an implication that indirect mbuf or mbuf having refcnt > 1 is read-only. What that means, all the entities which own such mbufs have to be aware of that and keep the principle as DPDK can't enforce the rule and there can't be such sanity check. In this sense, HW doesn't violate it because the direct mbuf is injected to HW before indirection. When packets are written by HW, PMD attaches indirect mbufs to the direct mbuf and deliver those to application layer with freeing the original direct mbuf (decrement refcnt by 1). So, HW doesn't touch the direct buffer once it reaches to upper layer. The direct buffer will be freed and get available for reuse when all the attached indirect mbufs are freed. > Though if you really need to do that, why it can be achieved by updating buf_len and priv_size > Fields for indirect mbufs, straight after attach()? Good point. Actually that was my draft (Mellanox internal) version of this patch :-) But I had to consider a case where priv_size is really given by user. Even though it is less likely, but if original priv_size is quite big, it can't cover entire buf_len. For this, I had to increase priv_size to 32-bit but adding another 16bit field (buf_off) looked more plausible. Thanks for good comments, Yongseok > > > > > > > > Does this make sense? > > > > > > I understand the need. > > > > > > Another option would be to make the mbuf->buffer point to an external > > > buffer (not inside the direct mbuf). This would require to add a > > > mbuf->free_cb. See "Mbuf with external data buffer" (page 19) in [1] for > > > a quick overview. > > > > > > [1] > > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdpdksummit.com%2FArchive%2Fpdf%2F2016Userspace%2FDay01 > > -Session05-OlivierMatz- > > Userspace2016.pdf&data=02%7C01%7Cyskoh%40mellanox.com%7Ca5405edb36e445e6540808d59e339a38%7Ca652971c7d2e4d9ba6a4d > > 149256f461b%7C0%7C0%7C636588866861082855&sdata=llw%2BwiY5cC56naOUhBbIg8TKtfFN6VZcIRY5PV7VqZs%3D&reserved=0 > > > > > > The advantage is that it does not require the large data to be inside a > > > mbuf (requiring a mbuf structure before the buffer, and requiring to be > > > allocated from a mempool). On the other hand, it is maybe more complex > > > to implement compared to your solution. > > > > I knew that you presented the slides and frankly, I had considered that option > > at first. But even with that option, metadata to store refcnt should also be > > allocated and managed anyway. Kernel also maintains the skb_shared_info at the > > end of the data segment. Even though it could have smaller metadata structure, > > I just wanted to make full use of the existing framework because it is less > > complex as you mentioned. Given that you presented the idea of external data > > buffer in 2016 and there hasn't been many follow-up discussions/activities so > > far, I thought the demand isn't so big yet thus I wanted to make this patch > > simpler. I personally think that we can take the idea of external data seg when > > more demands come from users in the future as it would be a huge change and may > > break current ABI/API. When the day comes, I'll gladly participate in the > > discussions and write codes for it if I can be helpful. > > > > Do you think this patch is okay for now? > > > > > > Thanks for your comments, > > Yongseok