From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 71377A0096 for <public@inbox.dpdk.org>; Sun, 7 Apr 2019 11:48:54 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C2A972BC9; Sun, 7 Apr 2019 11:48:52 +0200 (CEST) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) by dpdk.org (Postfix) with ESMTP id E32955A; Sun, 7 Apr 2019 11:48:51 +0200 (CEST) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 7011421E90; Sun, 7 Apr 2019 05:48:51 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Sun, 07 Apr 2019 05:48:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=mesmtp; bh=RvaSG2OvNBVz5UjrXk6L6uETfJvA2TWQqpEyj7209Oo=; b=gyrVqhXFaQj3 u+gJJUDHwMov1+M8votbzcWw5NCDx70PrfqRgveWJ7XC71z9c2hBOc4+h4o/7CAP onNLsVTU9rfeCwKTwO/gsL9auhOdy5PQDvZYAcNICamKC64lu3z/M5+7fJ/6Fl3X KgSa304ppSBn4NXKYRPg4hGbed42PEA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=RvaSG2OvNBVz5UjrXk6L6uETfJvA2TWQqpEyj7209 Oo=; b=r3MMk9d40iASWQ/DapdsVJhN2vuAja54o6/h5jPdG7oDD6LYo6OeZhzq0 N3D5JxWSYaqw/oXTujYAmbdXZssHOCujYfJP56V6nTkjXO0Iwix82e2VuGgfoUjp QihxL/jLFujELF1ysWJgSIFES7j7fNeFJ0KEjVlFif5CUUoD27ISKFIwHjxZCQDo tSf9rdndsJZWU9gRzg+MU1iknhCdwlUuL+my4jcCnEM9uspYjmGPmQUDf7+WXB+E bOd67Trsq0t1SH16kvGTCc5MIjK1geKV1tH6E++hmciXyGC0hQzdd9q7xtLKd2GF 3vX5d7JoJwdX3DypejOKIboFUgKtw== X-ME-Sender: <xms:gsepXLgy_8c7fs4-B-heZKJJj1-BXDHjgc58HISqfCuSTRkVVibt6w> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduuddruddugddvtdculddtuddrgedutddrtddtmd cutefuodetggdotefrodftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdp uffrtefokffrpgfnqfghnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivg hnthhsucdlqddutddtmdenucfjughrpefhvffufffkjghfggfgtgesthfuredttddtvden ucfhrhhomhepvfhhohhmrghsucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrg hlohhnrdhnvghtqeenucfkphepjeejrddufeegrddvtdefrddukeegnecurfgrrhgrmhep mhgrihhlfhhrohhmpehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtnecuvehluhhsth gvrhfuihiivgeptd X-ME-Proxy: <xmx:gsepXEfnDAAyUHpa-IFJgs7hrP9TnVTBF6ZIFxXk-cmIObYexRUP2w> <xmx:gsepXC3LYh32FN-Ex1xZTB933s1l5H6rzXJ_aVLZK1sno0EF50cfcA> <xmx:gsepXJJKXOXL679fnIFIYHshcZt52z8-QFXM1fEliwKjaYo_6U02gQ> <xmx:g8epXDVYCYCNAD1kKWyljzaawuXvhGWvYMOX0o5g2YVLBwGkIemsxg> Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA id D4FA210316; Sun, 7 Apr 2019 05:48:49 -0400 (EDT) From: Thomas Monjalon <thomas@monjalon.net> To: "Burakov, Anatoly" <anatoly.burakov@intel.com> Cc: techboard@dpdk.org, Ray Kinsella <mdr@ashroe.eu>, Bruce Richardson <bruce.richardson@intel.com>, dev@dpdk.org, Kevin Traynor <ktraynor@redhat.com> Date: Sun, 07 Apr 2019 11:48:48 +0200 Message-ID: <7166381.CkH77a7QuE@xps> In-Reply-To: <455a61b4-891d-eaaf-d784-2be884bcacbd@intel.com> References: <c0856556-a42e-d0cf-6a01-6279643c8089@ashroe.eu> <d9edf2cb-6894-b794-1402-0e1d11701deb@ashroe.eu> <455a61b4-891d-eaaf-d784-2be884bcacbd@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [dpdk-techboard] DPDK ABI/API Stability X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> Message-ID: <20190407094848.AiXw_rnRaenNh2kXqQeBAbq_qDo6tq6gnfn3DI-AGN4@z> 04/04/2019 16:07, Burakov, Anatoly: > On 04-Apr-19 1:52 PM, Ray Kinsella wrote: > > On 04/04/2019 11:54, Bruce Richardson wrote: > >> On Thu, Apr 04, 2019 at 10:29:19AM +0100, Burakov, Anatoly wrote: > >>> On 03-Apr-19 4:42 PM, Ray Kinsella wrote: > >> Actually, I think we *do* need to constrain the pace of development for the > >> sake of ABI stability. At this stage DPDK has been around for quite a > >> number of years and so should be considered a fairly mature project - it > >> should just start acting like it. > > > > I 100% agree. > > > > If you break your users stuff regularly enough, they will eventually > > start looking around for an alternative that doesn't break their stuff > > quiet so regularly. > > > > We often use the pace of innovation in DPDK as justification for ABI/API > > breakages, but that approach is a real rarity among the Open Source > > community. I can't think of any mature project off-hand that share's it. > > > > I would ask is Linux any less innovative because they offer a stable API > > and have an absolute commitment to never breaking userspace? Would Linux > > have ever been as popular as it is today it they broke userspace every > > quarter? > > > > They reality is that they (Linux) find workarounds and compromise > > because there is an uber-maintainer Linus who had a strong ethos from > > the start not to break their users stuff - we need the same ethos in DPDK. > > > >> > >> Now, in terms of features like the memory rework, that is indeed a case > >> that there was no alternative other than a massive ABI break. However, for > >> that rework there was a strong need for improvement in that area that we > >> can make the case for an ABI break to support it - and it is of a scale > >> that nothing other than an ABI change would do. For other areas and > >> examples, I doubt there are many in the last couple of years that are of > >> that scale. > > > > I would also be inclined to agree with Bruce's points on memory rework > > was somewhat of an outlier, we don't see many like it. > >> My thoughts on the matter are: > >> 1. I think we really need to do work to start hiding more of our data > >> structures - like what Stephen's latest RFC does. This hiding should reduce > >> the scope for ABI breaks. > >> 2. Once done, I think we should commit to having an ABI break only in the > >> rarest of circumstances, and only with very large justification. I want us > >> to get to the point where DPDK releases can immediately be picked up by all > >> linux distros and rolled out because they are ABI compatible. > > > > The work that Anatoly describes removing APIs that exposed internal > > structures and Stephen H's RFC similarly are good examples of the kind > > of work required to prepare for this change. We need to take a good look > > at the API and reduce the number of unnecessary internal structures > > exposed. > > > > I never expected it going to to be a big bang - but is a definite > > direction we need to move towards over the next few release. > > ...in this case, we have to think long and hard about the fabled EAL > rework/split, and in general *specifying* what is it that we want to > support, and the use cases that we want to target. Right now there is a > huge mountain of technical debt and kludges and workarounds that has > accumulated over the years, and it exists precisely because "every > change breaks someone's workflow". > > For example, just in memory subsystem alone, we have legacy mem, because > some use cases require huge amounts of contiguous memory, and not > everyone is using VFIO; there's all of the 32-bit related workarounds > and hacks; there's the single-file-segments stuff that could have been > the default if not for the fact that we support kernels that don't > support fallocate(); there are two different ways of doing in-memory > mode, because not all kernels support memfd's; there is a gargantuan > pile of workarounds (and "known issues", and just code in general) all > over the DPDK codebase just to support our multiprocess model and all of > the various warts that come with it. > > In fact, i would even go as far as to say that *most* of EAL ABI breaks > have been due to the fact that we store data in shared memory because of > multiprocess - so there is simply no way we can change these internal > data structures without ABI breaks, because even if they're not exposed > through user-facing API, they are still exposed by virtue of secondary > processes basically having an ABI contract with primary process instances. > > So, if we are to cement our core API - we have to make a concrete effort > to specify what goes and what stays, if we want it to be maintainable. > The DPDK 1.0 specification, if you will :) "DPDK 1.0 specification", that's a great project name :-)