From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50074.outbound.protection.outlook.com [40.107.5.74]) by dpdk.org (Postfix) with ESMTP id 0572437A6 for ; Wed, 25 Apr 2018 01:35:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=Q0aFt85CCoQwWbpFOxOe+19MXLunzRpQtQQbkPO9NrU=; b=CGaY8YUDtJD7hLc5YH9iYo4XRiwkq3ead8/4YvPKBppelD+UqvH4wXUmNo5WYRcxxeGGpa7WtxtPOz/DoKjbU2FGyTtIZZUjTnkAku6Bj3/JE1eFIGpPZ/ACIYi2GKC+0hJP3BT+MCezXKgRz1jmSjODlGQlCHVNvj7bvjGK/GY= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; Received: from yongseok-MBP.local (209.116.155.178) by HE1PR0501MB2041.eurprd05.prod.outlook.com (2603:10a6:3:35::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.696.17; Tue, 24 Apr 2018 23:34:57 +0000 Date: Tue, 24 Apr 2018 16:34:43 -0700 From: Yongseok Koh To: Olivier Matz Cc: Andrew Rybchenko , wenzhuo.lu@intel.com, jingjing.wu@intel.com, dev@dpdk.org, konstantin.ananyev@intel.com, adrien.mazarguil@6wind.com, nelio.laranjeiro@6wind.com, Thomas Monjalon Message-ID: <20180424233442.GC88208@yongseok-MBP.local> References: <20180310012532.15809-1-yskoh@mellanox.com> <20180424013854.33749-1-yskoh@mellanox.com> <934e714e-3cba-7f5d-9fcf-4f96611d758f@solarflare.com> <20180424160244.bggifhilvadxcjb2@neon> <20180424191538.exjgzoif4odhndew@neon> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20180424191538.exjgzoif4odhndew@neon> User-Agent: Mutt/1.9.3 (2018-01-21) X-Originating-IP: [209.116.155.178] X-ClientProxiedBy: CY4PR13CA0017.namprd13.prod.outlook.com (2603:10b6:903:32::27) To HE1PR0501MB2041.eurprd05.prod.outlook.com (2603:10a6:3:35::19) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:HE1PR0501MB2041; X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2041; 3:x2B3vpP2UywNjgHxPe80C91KJpRyVat3vSadFqdqlM6JanxKD2txEJFUr9GvKPWQ9Z5kE7/6ttat6gbLoIb+2iTtUiA3jT21r7L2Jflxbr3DRlK51Uyz6/WILF3ct/NLUSX1tFcysE5IIxo4XrGGxJ8D33SMnexcDlht30KU6MKAcGtOk1p0TrWeCjLKlooDVQ2CfZcusv+XLQCflsZ4anKHlKS3jqvVuJkGuhjyDUn6pESbB1jYPipBJY0CaB6n; 25:D7mr1xWgeLr176e+N9B9VrA4eK7sEp335aDPhkt193HiSp+MbHFN9DZ6kBjT4tprw9SNy2O9g7oTx6Lb1ic3o/2QV/wj3ogWUnUj+Ihue4UKcwIed40cbkaXOC4J6VGLVcl9Oki0Mtj55NCEJuvx5OxsMClIJhMel+KLZNoji+s9qmRa6xpmtGkmFrL4MWnUY0smqsI3+/R3uGUMgU4uEt9kgAycDuMIA4panAkhfA3RMcClIaPQJSi+nFhTV6x2Ukb2cJR6YU0cf+nQDrWHd5kHsR/6ZDnTeLXyA2Y1ofAvacXSMK8+vvxXuxtNIO9HLNLKQxIgGhtwH8rMFUW1/w==; 31:VmUPfjjTF+OsIch6RmawYHQzDj2AFwoLnj0dAiv9brlaODZrlTSOd/eKBANdWLbA/Sd471/UFDIUAnvZ9Z4uv3UZAPrMMVPKh/QY8KMwol3SMwdOJndxk/sEvmECEUNMLu0ZCpKKXg4EZZWwC+awYvmzl2ErIN33yeiFsiWncRVLorfLjHuCRLViO4ZW0wHzwfHBh2wy3V4Px9AFGpdBn2boiyZpxqx8FGcWW6ssgmc= X-MS-TrafficTypeDiagnostic: HE1PR0501MB2041: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2041; 20:MBiL183CYfGRFyRFkt5YihDTP7GTdEj2royKR0RfYyrxFfWkOeuoTpFA+OtgyR3aJI4nY+pCrZdZnyvENLM4RgkRBh4gSdc5KDi47Wr2lqu7lXW/KhPy0qDNNCCDHkA0z5ySB28G2MX0ejuhqa4CUXXGOdKCs0FbzTGHPaDVI+XcZKpldLtPoFqjdBA/qQYbIt0CZVozqMZNyHw9v9KMPQry+wXUhmctdulHLqQuhmu+g3w1e2l9G1908g+sGqjjdOvxA774wRPY9vtOFIV4Mm9EKjABSdHnGSsb2Oa0XrQnDbk8eVdptoYdWsDGsANEmI2thQ04tX1chSOFG3487WLXkx0440HPVD3+iEHPMo1/NK+c53yDEb6cCpVD8niMdqxXwUKv9uXZiHrYMJDTGktRDFlDmKT6MMfw3vTS1/QDtDmvDRy0inx5LS+8d/4oBT7BN2+7cDDFLgXfji2Q4UZigV9fJ8JAv7dp8nniNiDs/h3pjfxTFm/NlxVZ05xK; 4:zq0wf3T5+/t9A870I7qImZg6PrspGJi1z+xJrOuEEcDKxmrebtDlHmCbrze54QdTXpd3qHq+t4RQAG0n9h8yxoDRCeL7LmVA6cgWtnNYdbs/q7xS0Jy3CqHdCY82ng5zuEkt/HAuzX8iFXcV6RZPwEoGFw2pUlcgivILd6UrNmLBgYQscVzCFDEPT7658qz9EywIunvARP5anItgXaX2LipcD8BPt2d1HLLFKepi8wFp1rqwlFZD3QRA+tquLEHeorpx8VZa8q420owiqK50ww== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3231232)(944501410)(52105095)(10201501046)(3002001)(93006095)(93001095)(6055026)(6041310)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123560045)(20161123558120)(6072148)(201708071742011); SRVR:HE1PR0501MB2041; BCL:0; PCL:0; RULEID:; SRVR:HE1PR0501MB2041; X-Forefront-PRVS: 0652EA5565 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(366004)(39860400002)(39380400002)(346002)(396003)(376002)(199004)(189003)(50466002)(47776003)(68736007)(53546011)(386003)(6506007)(25786009)(23756003)(52116002)(97736004)(58126008)(54906003)(7736002)(81156014)(33656002)(81166006)(33896004)(561944003)(8676002)(105586002)(66066001)(6116002)(106356001)(7696005)(26005)(8936002)(98436002)(76176011)(478600001)(1076002)(6916009)(5660300001)(53936002)(2870700001)(2906002)(55016002)(305945005)(186003)(16526019)(93886005)(446003)(229853002)(11346002)(9686003)(6246003)(86362001)(4326008)(956004)(316002)(6666003)(5890100001)(486006)(476003)(3846002)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0501MB2041; H:yongseok-MBP.local; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?iso-8859-1?Q?1; HE1PR0501MB2041; 23:NzcYRk4uxDIDhJIPZvfB6/SExGkFo2rK+HrvV?= =?iso-8859-1?Q?/NKGQoLHC5GK4j9cORKtgKUbsMRm7aPxVwPJDKKyurT4cXJTrjHHGWGveH?= =?iso-8859-1?Q?NlJbPnzbSOsp6O9ZWkt0OpRRRBlT4+DQ/cw9O1BN2ZGb+f8c58FIbf5US9?= =?iso-8859-1?Q?p9vGdItJP0fTus+DxahyMg8yAkTdflg8+PmvSIwixmKya38C8pqgglNQfw?= =?iso-8859-1?Q?qTDE2YpJU1nkZ919p1GnjZyrV/bAm2qIXA4Jk+uvb1P3W1d5xCBT5Eodn+?= =?iso-8859-1?Q?SEF8TTW+xp5lUdjGhKIHaNhHQNTrMyXuEb4QRV/INhDcrwZdNw+gbdsk4L?= =?iso-8859-1?Q?dviQFxaDT/4uT8tzyN2YM+pVsOglds9+1Te4zZqKd0mJNwcusBrWgsf12m?= =?iso-8859-1?Q?3aCmV7Gl6F8MFhI9ixrZI+aGoarlpPr9ZLa2qqGsL69HXO/r2xXUySbY/c?= =?iso-8859-1?Q?TAoUNeqg31t4Z2guvSm5eBkFhW2OfjiVoPNIlrXL/1YfHCpij3awek4hup?= =?iso-8859-1?Q?dZ56ZLivl7ALYpnaq7S4wX6YbyMKcGZGpTo54O9Y1ZcNfVrtyE866nXkhc?= =?iso-8859-1?Q?eoIaGc4RDD8D/2PVRuGZubk1bR25OFyiSk5HtFx/Ix1uoU6h1Ki82h8FWk?= =?iso-8859-1?Q?dUY48dA3fpWLvDO/lUObcmDTOhmJUFm/RPSdzybFvDvC16k9d9+jHYoSua?= =?iso-8859-1?Q?zL5gZfBb+GnTSabgG+cKFcG4rqYoO4ToqePY/4oFqMvnGd76jAMZ/XDlk4?= =?iso-8859-1?Q?e4lzXWDOUNbYgGHg0e3ULYaTiL6lfCoAO5mAF5edpjq/tbH4Q+h1RqtI6I?= =?iso-8859-1?Q?9HXsYIXYL9LXG+BmJVo2CGhTov2H9soy3Q55I/4iGVmH5giXNAhCQc72uO?= =?iso-8859-1?Q?ZqEVyT4eVUNJ/LWpLMRdvfYPLVtP7EWghlf9yoaCRGwYVSEFr0EBt0lV2G?= =?iso-8859-1?Q?hUgYYEK5Zu+C/Z0JSCfKpUKgbtoha8zt7zx0LqAcbvHkaGjb7wnctPZ8bX?= =?iso-8859-1?Q?hCGwxPWqMjnO065DWC6GflzLjAIsMCpeu0EX/eOyZTs++B/bNGzg4SY0I2?= =?iso-8859-1?Q?MhYlk4YguDB0ULE3mGPF+OkCDjzQ12XLqhttEVmKM5JCv1dL/SHX5Dlteg?= =?iso-8859-1?Q?fRk9WfqHj+488gFjd4qUL9i81iypC/gI3QMo0xKQtmnKNeAwhSC7HK9HLV?= =?iso-8859-1?Q?Z7T8AaEtPZvOOoz9EvF1AGBfG8ibYuS6JPJ8vnyO0THFw/Mwa14/V9IHNr?= =?iso-8859-1?Q?ehcMsSrbuClqaeUwUlMbglL1xeBnPkQZkFHdVaoAa7rdw3TnD31xVojiFC?= =?iso-8859-1?Q?w0Z9XuDGcewSyqSk4g+kOS2VuAhL0oWJ1LowHQcSRxIPkHOsO671mewliC?= =?iso-8859-1?Q?MV6Z9rVmGjXjEBAInOQiieltzWegdhjZUzAW0vJ8Nm0QweniG7gPR92K4L?= =?iso-8859-1?Q?thGi6iRhWPaA10LGd4HmozAdbO34ZowGbeO6f?= X-Microsoft-Antispam-Message-Info: 4r65RyNo8AAfDAZtbV3vono8XZGl3ndkM3if240k5EICYlEvZwfeD6N1Q4RxIuW+8rXXcEqBO7DxliIcvyM/Z5oLoyiNkJ9NLcweNyVbCtzT5MmCJ/heStoWfN/iT2+lk0jmWqMTZuFTNdtVDT9HmKho13mdltvsx+dSMCA5buV7VpwuJzjbZIW5/wqX5VoM X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2041; 6:myJ3lAEzvc5Hajl5AbH5eqyqLjK6GliEO33DsAmPvMDtbBfKWHV/1a1yxBQ3TvVEwMafqlcTwgD5Dcy8BLr6klIpvWwfvIzquw93ANMHrR1UfqAqC4p4Zij4j75S2SdOZRWTIXydY9mil48PiaSr9vgNzcsNNhJgQkbjlqo7hsM4EWbKNnYuHjge0ysp+MOfBCJvNDARp74rGLwOtkaxHmNk0feVOYm2inRweUStetScRYlqXlYxCGaRdQ8DZL+WFgsL9gZezJYGFDNN+NBhv2p+egki0e5Mk25+3cbJEraYMR85XlkJeqjAUugEdIZMRGgKadicqYHvem6s6WxqXqEMA2uOBS7bSJtZv0pv/zDHwVoYwXUdLkRO5rTaDYknPQXPddRvueOpet8YHwNmADx0NU1AvP2WKSZU9n62RD0LH9+5kD5faxuEX8hk+jcC30PysEUI0MKijHCuXBVY7Q==; 5:lfj7Ady2F1CYe6XwjkreceaUrBkYMwVTm/0s7YIZAff3m+xkUPQAteG3Dvq9RSZP8zCQjZqn3+t9a1iTayVtk6sjFTwcXHgciGujV0I+LqqwTaZaG6V3FeqX1EM7Vfv2UibweBYtT17orCDdoOpswPpnwaBLQm29hXZPeD/8RAY=; 24:btVkGjiBzSxjAjon6WEU++YvMnbVVRRaaIJVRq0UNQPfx+MZQIajdfeEMbZs0ooDo/QEiaSQboE49nVX51NBftjEAqm+6nUdLj6eC4IiTZQ= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; HE1PR0501MB2041; 7:65XzwPwrMnrvDTI5fR4fpzYcY3xId0wtD1HMrVEHRGFPu4Rj3XxIvRi83GOohYiJt/CHLvGABa4c9rZt9pJw1h2mWgixGMTbRcEMa0T+2U3cf94jtha2ZbF6ukx5GiiwrpEMtpJtmAIFSKP6nF1I8oTrNW6SBL+QED5GUG8wMRuJ+jzcuX3zkjz+SysnQ6hic3Oyxq0zm6LuDe/72ZuaVIxZfARh0Pd0heQsGQblMeVz6u2GqWqHYm8AYb9aH0wE X-MS-Office365-Filtering-Correlation-Id: abc0f73c-45f5-430b-ca3a-08d5aa3bfeda X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2018 23:34:57.7554 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: abc0f73c-45f5-430b-ca3a-08d5aa3bfeda X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0501MB2041 Subject: Re: [dpdk-dev] [PATCH v4 1/2] mbuf: support attaching external buffer to mbuf X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2018 23:35:03 -0000 On Tue, Apr 24, 2018 at 09:15:38PM +0200, Olivier Matz wrote: > On Tue, Apr 24, 2018 at 09:21:00PM +0300, Andrew Rybchenko wrote: > > On 04/24/2018 07:02 PM, Olivier Matz wrote: > > > Hi Andrew, Yongseok, > > > > > > On Tue, Apr 24, 2018 at 03:28:33PM +0300, Andrew Rybchenko wrote: > > > > On 04/24/2018 04:38 AM, Yongseok Koh wrote: [...] > > > > > + m->buf_addr = buf_addr; > > > > > + m->buf_iova = buf_iova; > > > > > + > > > > > + if (shinfo == NULL) { > > > > > + shinfo = RTE_PTR_ALIGN_FLOOR(RTE_PTR_SUB(buf_end, > > > > > + sizeof(*shinfo)), sizeof(uintptr_t)); > > > > > + if ((void *)shinfo <= buf_addr) > > > > > + return NULL; > > > > > + > > > > > + m->buf_len = RTE_PTR_DIFF(shinfo, buf_addr); > > > > > + } else { > > > > > + m->buf_len = buf_len; > > > > > + } > > > > > + > > > > > + m->data_len = 0; > > > > > + > > > > > + rte_pktmbuf_reset_headroom(m); > > > > I would suggest to make data_off one more parameter. > > > > If I have a buffer with data which I'd like to attach to an mbuf, I'd like > > > > to control data_off. > > > Another option is to set the headroom to 0. > > > Because the after attaching the mbuf to an external buffer, we will > > > still require to set the length. > > > > > > A user can do something like this: > > > > > > rte_pktmbuf_attach_extbuf(m, buf_va, buf_iova, buf_len, shinfo, > > > free_cb, free_cb_arg); > > > rte_pktmbuf_append(m, data_len + headroom); > > > rte_pktmbuf_adj(m, headroom); I'd take this option. Will make the change and document it. > > > > > + m->ol_flags |= EXT_ATTACHED_MBUF; > > > > > + m->shinfo = shinfo; > > > > > + > > > > > + rte_mbuf_ext_refcnt_set(shinfo, 1); > > > > Why is assignment used here? Cannot we attach extbuf already attached to > > > > other mbuf? > > > In rte_pktmbuf_attach(), this is true. That's not illogical to > > > keep the same approach here. Maybe an assert could be added? Like I described in the doc, intention is to attach external buffer by _attach_extbuf() for the first time and _attach() is just for additional mbuf attachment. Will add an assert. > > > > May be shinfo should be initialized only if it is not provided (shinfo == > > > > NULL on input)? > > > I don't get why, can you explain please? > > > > May be I misunderstand how it should look like when one huge buffer > > is partitioned. I thought that it should be only one shinfo per huge buffer > > to control when it is not used any more by any mbufs with extbuf. > > OK I got it. > > I think both approach could make sense: > - one shinfo per huge buffer > - or one shinfo per mbuf, and use the callback to manage another refcnt > (like what Yongseok described) > > So I agree with your proposal, shinfo should be initialized by > the caller if it is != NULL, else it can be initialized by > rte_pktmbuf_attach_extbuf(). Also agreed. Will change. > > Other option is to have shinfo per small buf plus reference counter > > per huge buf (which is decremented when small buf reference counter > > becomes zero and free callback is executed). I guess it is assumed above. > > My fear is that it is too much reference counters: > >  1. mbuf reference counter > >  2. small buf reference counter > >  3. huge buf reference counter > > May be it is possible use (1) for (2) as well? > > I would prefer to have only 2 reference counters, one in the mbuf > and one in the shinfo. Good discussion. It should be a design decision by user. In my use-case, it would be a good idea to make all the mbufs in a same chunk point to the same shared info in the head of the chunk and reset the refcnt of shinfo to the total number of slices in the chunk. +--+----+----+--------------+----+--------------+---+- - - |global |head|mbuf1 data |head|mbuf2 data | | | shinfo|room| |room| | | +--+----+----+--------------+----+--------------+---+- - - Thanks, Yongseok