From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on0043.outbound.protection.outlook.com [104.47.0.43]) by dpdk.org (Postfix) with ESMTP id A62457CF9 for ; Wed, 25 Apr 2018 19:40:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=DErKeIBryPJIiyKBDdv1grGe6RWP1PeZXa3K2VYf1HE=; b=VECF/d0VBCG6T2v9fC/4FQlsVTgVqvkTbwjzqVBXO4OuSiIXj0EutzqB8M3YhvNyGOSnEpEJ47l+4TV8npoera/vLvZT3wxOVnT6nuhUiJLocprvNX09oa+jBEFfyHZKF/gIpxjIbg2xyQRMT/w+RWhcZsey8L04h+Jw9oaT4mM= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; Received: from yongseok-MBP.local (209.116.155.178) by VI1PR0501MB2045.eurprd05.prod.outlook.com (2603:10a6:800:36::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.696.13; Wed, 25 Apr 2018 17:40:21 +0000 Date: Wed, 25 Apr 2018 10:40:06 -0700 From: Yongseok Koh To: Andrew Rybchenko Cc: Olivier Matz , wenzhuo.lu@intel.com, jingjing.wu@intel.com, dev@dpdk.org, konstantin.ananyev@intel.com, adrien.mazarguil@6wind.com, nelio.laranjeiro@6wind.com, Thomas Monjalon Message-ID: <20180425174005.GC3268@yongseok-MBP.local> References: <20180310012532.15809-1-yskoh@mellanox.com> <20180424013854.33749-1-yskoh@mellanox.com> <934e714e-3cba-7f5d-9fcf-4f96611d758f@solarflare.com> <20180424160244.bggifhilvadxcjb2@neon> <20180424191538.exjgzoif4odhndew@neon> <20180424233442.GC88208@yongseok-MBP.local> <6f520eb1-8ce4-16bc-49b5-ddd25f73f330@solarflare.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <6f520eb1-8ce4-16bc-49b5-ddd25f73f330@solarflare.com> User-Agent: Mutt/1.9.3 (2018-01-21) X-Originating-IP: [209.116.155.178] X-ClientProxiedBy: MWHPR15CA0058.namprd15.prod.outlook.com (2603:10b6:301:4c::20) To VI1PR0501MB2045.eurprd05.prod.outlook.com (2603:10a6:800:36::19) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(48565401081)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:VI1PR0501MB2045; X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2045; 3:OozL1CLbWithXXn//iYqSe5CajoRuwCoP2A1+gmcD12+ZMMv1a/JyOqmukHgZHipXLwGvBcvtfSvWjE/xlvnomxy63Ldn/0JTz7lZJAl5A7FrZPxWRY9xbeJL0sEHKqlQAYiTk12bApZoA+z90NcrEt92wE2jfpZpFRNWAOBrvFSZr3oPemfE56OW+uF/wfspB9Lq1QHtVgPM1XQX6N2d+dbfd966SfBhPpEbEehHcQZ5487SbtAKBMTNzjriLMm; 25:20WgZq8Z9fw7MQhzYrmnqiz/NIhDk2xRKjcyqsAzXS45npU9tki19RpXVI7RPeosjcwLDqW+vXHeeGVNrWdfZAyanG178b6e4oquJWvFdyvLnQxp1u3HT5DXPIsGXHjb7FNkJNHp44C34K817PYOWOIwKW/CZ8fWYe/gqd3RQ31V81xf91fipBMPZJxcLkqgXVMBdtTCRh1MgXVCONjYXKHPu1lXI6010WcN+qGbzLgeGlTm0lpY0c4xV7OR1p51+yBMt26rRYgdOqf5ROLjhtqMG3YxyEGg6mgQehVmdllMT6ikEt8hIX///M10DqMs3fuL45A3osS+huoCYA15Hw==; 31:JQRJpCYFY95zpmWHvmw+uN+GlEykyNkiiFgCA16tYyiiJCAH7X+sPXZYOkFYBLhysI4JTqK92P5bHgZrIMedif0dkrY0UafpHEX2IN8BH2oOLS8uoaLnF6YnsicyPGn8ijs+/4IubUvaKzfjW9bZRz1Ll4lgGXibVa2pmZdw/Hy0/FvyxGywP50Ug0+ZCqpt7jGnwcMKsnhtFLrqiBK8Xnr0y5sQZ4fxdXJT5NJGI7g= X-MS-TrafficTypeDiagnostic: VI1PR0501MB2045: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2045; 20:GtfV23988Zj7WN79DgYAoXd7N+0k0HJ7h38MAvrLigOeIHtUGZcMM2y9qK2QZ2w/HAvtuvRfjW9rfznhuCLm28Y7HOsc+pFOicuKW/dk1j9eQGO2uTZtH99MgyO7X89OYng71x++SoAToGMmx/MbSORZuQ+TbieLaVFkwVj582NavbRrGtWsFSNWoIMrLThmIkgR5z+Cl0q0vriNpXygvozFfHGX9hMPUpnbHjHaov8OUbPIMHOKztZ9F3Vqdx3Gq2fmC2gsxXXkI0XLujw4SGSI5Jdgb5pkoKl55qpxujStLDFDVYx0BaBuAa/5DNtiJ0MMA9rd7G+OkT/L6R1A6gq5KY22xYRszzsf5GTZWJoss4H0s/+zylxmiMedi73x19DOJSA6Kr3fK2EPEKA8ReGL0kOuFEy1AB7M5/QEzzrRMqfUWqIxppBme8e9ZOUXxaHN1j/J7TgYqApACg0hDDbzoHogVOMY7hGB2uh/2na5hsRNyYZ1yyK1hFzdi+nB; 4:umDSbGFINx7KbFGFUEgDb/dJrSy5avB3CukjpZHqZUDVTrZNX1VEv6obDhT1/xP4j9QUzwXtP7u7unjLXoE5UU6WWej7dOIDxtT4VlJAm9LtKcOOTspcMB6b2zplnw8GNi8m3b+zcSp4KXrACCLsS3rUHiw5h7dOnUawy+I4Rsr12+P3W159bJlx3lPf2+FOMv8QpaiWhMjCPAy6/A68U9uUw8Zsch3MMHH/QaMlUZvkFBdkwGTbZ7VOJW1ie7AhVSl1r5ZHzfstTIeGwXS/iA== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3231232)(944501410)(52105095)(3002001)(10201501046)(93006095)(93001095)(6055026)(6041310)(20161123562045)(20161123558120)(20161123560045)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011); SRVR:VI1PR0501MB2045; BCL:0; PCL:0; RULEID:; SRVR:VI1PR0501MB2045; X-Forefront-PRVS: 06530126A4 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(396003)(39380400002)(39860400002)(366004)(346002)(376002)(189003)(199004)(81166006)(956004)(478600001)(8676002)(2870700001)(9686003)(55016002)(186003)(93886005)(53936002)(16526019)(229853002)(86362001)(25786009)(305945005)(66066001)(5660300001)(23756003)(54906003)(6916009)(6666003)(446003)(58126008)(476003)(2906002)(47776003)(486006)(11346002)(97736004)(5890100001)(3846002)(6116002)(8936002)(68736007)(316002)(59450400001)(6506007)(33896004)(50466002)(106356001)(7696005)(52116002)(26005)(53546011)(1076002)(76176011)(98436002)(6246003)(4326008)(33656002)(105586002)(386003)(7736002)(81156014)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0501MB2045; H:yongseok-MBP.local; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?iso-8859-1?Q?1; VI1PR0501MB2045; 23:EpwS8LtiG0Xd6oRZ63ySz46/NHsMo2DHpGiMY?= =?iso-8859-1?Q?NXEs3r0m+eeQcHMLMSoTVEDQyNSGAuwj6+nlwwRxTzKzoYabvFc2O9SUwG?= =?iso-8859-1?Q?MdULPEllEGNWQdE+KJAEM9PKrenVHLPrfRbp42b4807f0HZvuUiyYCwnDN?= =?iso-8859-1?Q?P3CPhU+sp7f4UFoqELsSIxkb+8dlWP3b3LEytPXi1gGXnUQELYJJN5pSe5?= =?iso-8859-1?Q?jeFLYD+LcduvXBk6jhg9w2TFMef1t6CFTAzgWNLTZezAUPrh4ISrqlw/dD?= =?iso-8859-1?Q?6fqIQYrpqTCoZc1azMyX8qnql+SNqjIypI1b325V0m4QFQviVih9GuOreq?= =?iso-8859-1?Q?6R8nbsASzh9L0Y9j4TUEQad24BJBAACH+ACJaG5H4JYvvPzePu+iDQdo9N?= =?iso-8859-1?Q?gCZ0CQGHUjJ6sqx5/RY4JBf9ln3Bi8RGU9vr2PfGYZYIke0itpr/zPc4/q?= =?iso-8859-1?Q?QPKf3Vd6+lEoCuqtvwAY7fEIAcImnNBgdoZaF1sZgV9VAVyQM5WCLFsFhT?= =?iso-8859-1?Q?vY9Qm353xQ/Ad6hEGcLOjyXffFJnO4EdE98JrR39mIAzkmsUXrrDS6+cW3?= =?iso-8859-1?Q?O6P/TZg42Z1J1QzkUazncCC4ZY+TJlB0rhQy6FzktJPkvlUMPWA7wlk092?= =?iso-8859-1?Q?j4hIh08MxfINJZIoBgCPbVhpkwyiACpAj58CnSd1m2E3Iep0SxHbUhqJJr?= =?iso-8859-1?Q?yGX6C+JUcfpU/YsdC4RTAyNWYSunDJsjMH4u/7U9nN2Hi10I23gZb7SlZP?= =?iso-8859-1?Q?dF8H48XVAx8S6eBArx+ailVg2A8KFLEIEwXFI+b/EQnQMrPbSQbsBLYSLX?= =?iso-8859-1?Q?Z/2YJE0r2KfE1xM8DCJfl3IllQh9bEtDT6EvgLe4rV9KRJ8xo57a+7fHFm?= =?iso-8859-1?Q?7dzbo3oGSq0ISXw5t3clwuojrB1Ebt/khnuY3C7Usxl4rwP0+bXFYleopd?= =?iso-8859-1?Q?qLjLo9GsnzOGzGAGBN3UlZbardb1ivWkSXc+361RoKV/O+wf4xKf8L/9s7?= =?iso-8859-1?Q?y1VrKMMnGveqiroqwEKYKPqC00I1XRfKSrKzAEEmGqCL1A/hoQhC/yRzoE?= =?iso-8859-1?Q?NwHd/bEf7Uy8JcKLkCUbqT4inFHlnk3RutdaisRgYK5KOwn3j1lF6oaTgv?= =?iso-8859-1?Q?ZXEFd4rc4DEwDX2rsh4oHFjhZeZJ+1Reg6t1t4pDsXbiOy1uxvCIPOGgVP?= =?iso-8859-1?Q?VvDDiOB0mVLf5bNXegLgCT163IeggXmVlp7IyPAAPhicRLVKZdxcylHwdF?= =?iso-8859-1?Q?ktv4NzHLo/EDM42QdQecKNysfT5o0dgra9D4Huaa55hYBe3/D36hIHaYRs?= =?iso-8859-1?Q?pRPNHqW9Lx6VppS4IPIdIi44a0UbMgNlSB1rNVj81lB8nhaFxNdE8Ii3oz?= =?iso-8859-1?Q?lkp1D8PSUZDYY0rn7JXZbxMBGN5j9Fc0yXvd7/Q2fqyunmGA7C/Fq3CMpO?= =?iso-8859-1?Q?GWxRF+V/W2BaNg6FsI7uHtq3bD3ngLscO52yzK4IL9GQTQjmd6aaeEOEKt?= =?iso-8859-1?B?UT09?= X-Microsoft-Antispam-Message-Info: spBVP3t3Haz3XbBNce0iXRAo7EByskS5kpi0x/4S7X+ZDXwCT7/BB0UT4pNT+M754k6w6atPH6gAu6HSihmYjFpx/biU9EsM402rkFgGwGUeDXTDyND+PNQqx5SajZ5lXMnQlIMxgPks3kN3CsoD0uAvGzkBgFlQNJtoimVg1HA3uAge29EUcUvCmerfXP8/ X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2045; 6:XjHzphDcx9iTjXNUDWYNfiUJEMWpgGomVQIBkU9Dm1bjV4q5kAu0dKm5jDOVEKgzCf6W+t+dpQT0uE0ENtVp60I8SPSadVKnExc8Kemk/6WUpIOPc6wsp4oakm/4MLuHjzWgcRHdeVzHnccIOOJBMKzTa3OtjDqI3OzxlK+8iWqxXotjGFg8GIBhNHjAP2/EGorVMiqF902tuy0nZrsCJKMT8i98sfqSRGBDBwFt/6nTxb+9WUljPrwNPEihJWI+EEMGl8LNy71eAv1qEEEnIw/hKx07gFfcJ8Kd7x+pf2jqi3Gex+a+2nEsMoVbP1nmrkTkAPxG6xmLmWYtJh/n6cDY10/sXlIyWdbcuWxvuYSeBEWPgv3HZ2wZK8sSMaqRS8CW4smb5Z36FLKrqtsDdn91Psc7mB3CLGwkGvQgJUdW1As8qFmgy+VwJF1+Cr3N/P93s1klDC3sYQMLvu62iA==; 5:W04A8B+fhNXUlFONqdZvfSK+cQsHQTgAJs/JU2jXu+48ioJWW4xl01wAdmBftaa2VM1uyYPMn3QLe2JzIGIPBxhOvFB1IINo5YxwCB/YJUa2l6bMVgZ4jVgX1zSAhFOkgqqmzwsfbwCA88+yTygOlgFuAGUxxx6Wiw60+4oj/MQ=; 24:G067U6r1szq+0ulmHOeQfYpLWTMiHIbrFwpsAPwFXjW9WkTge9jTP4tujBaCuULTGZL6fxzOt3e2hbHNA8HtJtHpzmRIb+Wn3h8b7z7WWQw= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2045; 7:UEg4AHzBX6C8rR+IKCSG9Xzv4lIi0XjROeID5eCXfAh4TFT68EYJS7PhAWnsZhJ5B71jBtjwteE0W7w/IgMnX4NF1m09Lbk7Kijs1NBQ7mTkjjse4KXAir7Bbs24mYP2CnGYCfFEJMoq3+OudEG7jh7idpiLUUux6HDpt+HIuChw5tdg41Qn/4KjNlq+jJm91TZ86d5Z8r4dx5OqD9rC+7BqMi+ihc4H/KiqsW/1Fag8PeUNqG/7w+wBnbyXg7fD X-MS-Office365-Filtering-Correlation-Id: c4baede1-60f7-424f-3b1b-08d5aad39f7a X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2018 17:40:21.3201 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c4baede1-60f7-424f-3b1b-08d5aad39f7a X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0501MB2045 Subject: Re: [dpdk-dev] [PATCH v4 1/2] mbuf: support attaching external buffer to mbuf X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2018 17:40:27 -0000 On Wed, Apr 25, 2018 at 05:45:34PM +0300, Andrew Rybchenko wrote: > On 04/25/2018 02:34 AM, Yongseok Koh wrote: > > On Tue, Apr 24, 2018 at 09:15:38PM +0200, Olivier Matz wrote: > > > On Tue, Apr 24, 2018 at 09:21:00PM +0300, Andrew Rybchenko wrote: > > > > On 04/24/2018 07:02 PM, Olivier Matz wrote: > > > > > > > + m->ol_flags |= EXT_ATTACHED_MBUF; > > > > > > > + m->shinfo = shinfo; > > > > > > > + > > > > > > > + rte_mbuf_ext_refcnt_set(shinfo, 1); > > > > > > Why is assignment used here? Cannot we attach extbuf already attached to > > > > > > other mbuf? > > > > > In rte_pktmbuf_attach(), this is true. That's not illogical to > > > > > keep the same approach here. Maybe an assert could be added? > > Like I described in the doc, intention is to attach external buffer by > > _attach_extbuf() for the first time and _attach() is just for additional mbuf > > attachment. Will add an assert. > > Not sure that I understand. How should the second chunk with shared shinfo > of the huge buffer be attached to a new mbuf? Okay, I think I know what you misunderstood. This patch itself has no consideration about huge buffer. It is to simply attach an external buffer to an mbuf. Slicing huge buffer and attaching multiple mbufs is one of use-cases of this feature. Let me take a few examples. rte_pktmbuf_attach_extbuf(m, buf, buf_iova, buf_len, NULL, fcb, fcb_arg); |<------ buf_len ------>| +----+ +----------------+------+ | m |-------->| ext buf |shif | | | | |refc=1| +----+ +----------------+------+ |<- m->buf_len ->| rte_pktmbuf_attach(m1, m); +----+ +----------------+------+ | m |-------->| ext buf |shif | | | | |refc=2| +----+ +----------------+------+ ^ +----+ | | m1 |---------+ | | +----+ rte_pktmbuf_attach_extbuf(m, buf, buf_iova, buf_len, shinfo, fcb, fcb_arg); |<------ buf_len ------>| +------+ +----+ +-----------------------+ |shinfo|<--------| m |-------->| ext buf | |refc=1| | | | | +------+ +----+ +-----------------------+ |<----- m->buf_len ---->| rte_pktmbuf_attach(m1, m); +------+ +----+ +-----------------------+ |shinfo|<--------| m |-------->| ext buf | |refc=2| | | | | +------+ +----+ +-----------------------+ ^ ^ | +----+ | +------------| m1 |---------+ | | +----+ > > > > Other option is to have shinfo per small buf plus reference counter > > > > per huge buf (which is decremented when small buf reference counter > > > > becomes zero and free callback is executed). I guess it is assumed above. > > > > My fear is that it is too much reference counters: > > > >  1. mbuf reference counter > > > >  2. small buf reference counter > > > >  3. huge buf reference counter > > > > May be it is possible use (1) for (2) as well? > > > I would prefer to have only 2 reference counters, one in the mbuf > > > and one in the shinfo. > > Good discussion. It should be a design decision by user. > > > > In my use-case, it would be a good idea to make all the mbufs in a same chunk > > point to the same shared info in the head of the chunk and reset the refcnt of > > shinfo to the total number of slices in the chunk. > > > > +--+----+----+--------------+----+--------------+---+- - - > > |global |head|mbuf1 data |head|mbuf2 data | | > > | shinfo|room| |room| | | > > +--+----+----+--------------+----+--------------+---+- - - > > I don't understand how it can be achieved using proposed API. For the following use-case, +--+----+----+--------------+----+--------------+---+- - - |global |head|mbuf1 data |head|mbuf2 data | | | shinfo|room| |room| | | +--+----+----+--------------+----+--------------+---+- - - ^ |<---- buf_len ---->| | mem User can do, g_shinfo = (struct rte_mbuf_ext_shared_info *)mem; buf = mem + sizeof (*g_shinfo); buf_iova = get_iova(buf); rte_pktmbuf_attach_extbuf(m1, buf, buf_iova, buf_len, g_shinfo, fcb, fcb_arg); rte_mbuf_ext_refcnt_update(g_shinfo, 1); rte_pktmbuf_reset_headroom(m1); buf += buf_len; buf_iova = get_iova(buf); rte_pktmbuf_attach_extbuf(m2, buf, buf_iova, buf_len, g_shinfo, fcb, fcb_arg); rte_mbuf_ext_refcnt_update(g_shinfo, 1); rte_pktmbuf_reset_headroom(m2); Does it make sense? Thanks, Yongseok