From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7E0FDA04DB; Tue, 10 Nov 2020 15:34:55 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 85ECD2B93; Tue, 10 Nov 2020 15:34:53 +0100 (CET) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 45A21F90 for ; Tue, 10 Nov 2020 15:34:51 +0100 (CET) IronPort-SDR: I0hvbIMK6YXTApa8MFYrRqT18/iKbcAeP27D3iU49RFPTXCwya3uGIuCnsURjM8ORbDCubEl0M eMUlyfB0W+iQ== X-IronPort-AV: E=McAfee;i="6000,8403,9800"; a="231607506" X-IronPort-AV: E=Sophos;i="5.77,466,1596524400"; d="scan'208";a="231607506" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Nov 2020 06:34:48 -0800 IronPort-SDR: PcfDzeVVbGJtTdnHYlFXoC/C7kyM5+LYmiwf/b022WFQC+OLHIlTpr9HUiii3WJRubwnNNbhXJ BaYUWmymQq5w== X-IronPort-AV: E=Sophos;i="5.77,466,1596524400"; d="scan'208";a="541343370" Received: from bricha3-mobl.ger.corp.intel.com ([10.213.241.186]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 10 Nov 2020 06:34:46 -0800 Date: Tue, 10 Nov 2020 14:34:43 +0000 From: Bruce Richardson To: Thomas Monjalon Cc: David Marchand , "Jiang, Cheng1" , Maxime Coquelin , "Xia, Chenbo" , dev , "Fu, Patrick" , "Yang, YvonneX" , "Yigit, Ferruh" , "Hu, Jiayu" Message-ID: <20201110143442.GI1641@bricha3-MOBL.ger.corp.intel.com> References: <20200910064351.35513-1-Cheng1.jiang@intel.com> <20201110111930.GC1641@bricha3-MOBL.ger.corp.intel.com> <5954084.PnIyprfhLy@thomas> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5954084.PnIyprfhLy@thomas> Subject: Re: [dpdk-dev] [PATCH v10 0/4] add async data path in vhost sample X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Tue, Nov 10, 2020 at 02:37:02PM +0100, Thomas Monjalon wrote: > 10/11/2020 12:19, Bruce Richardson: > > On Tue, Nov 10, 2020 at 09:17:36AM +0100, David Marchand wrote: > > > On Tue, Nov 10, 2020 at 4:02 AM Jiang, Cheng1 wrote: > > > > > - This series breaks external compilation, as the external Makefile was not > > > > > updated. > > > > > > > > > > > > > I'm not sure I understand what you mean by external Makefile, > > > > because as far as I know, makefile has been deprecated. > > > > > > make support is dropped for dpdk compilation itself. > > > > > > For the examples, the users will use make to compile them, as this is > > > the only way provided to users *out of* dpdk. > > > But the examples are compiled too via meson when compiling dpdk itself > > > if you pass -Dexamples= options. > > > > > > > > > Bruce, > > > > > > I want to avoid more misses like this one. > > > > > > If we want to keep the examples compilation in meson, we need a > > > consistent framework to compile in both cases. > > > Right now, we don't export meson for examples, and it makes no sense > > > in their current form. > > > It seems simpler to me to only maintain make support, and meson can > > > still call make for each example. > > > > > > Another solution is to rely only on test-meson-builds.sh, but then it > > > ends up on the maintainer shoulders, so I prefer the solution above. > > > > > > Other ideas? > > > > > > > Hi David, > > > > I've been thinking a bit about this since I got your email, but inspiration > > for new ideas has yet to strike me. > > > > While I can see the downside of having both make and meson build files for > > the examples, I'm loath to see one of them dropped. Here is my current > > thinking on it: > > > > * We obviously need to keep the basic makefiles so as to make it easy for > > end-users to compile up and use the examples separate from DPDK itself. > > The makefiles serve as good examples as to how to pull the DPDK info from > > pkg-config. > > The external compilation is part of the example, yes. > > > * On the other hand, being able to build the examples as part of a regular > > meson build is a big advantage over having to build them separately, and > > allows errors with examples to be caught quicker. > > In the past we had a Makefile which builds all examples. > If you want, it can even been called at the end of meson DPDK compilation. > Yes, we can do that, but my concern is not so much about having them built as part of build tests, but rather having them part of a build workflow for developers during development. Thinking here in particular of those coding with IDEs such as eclipse or VScode, which I know a lot of people use - including myself from time to time, depending on what I am working on. > > It's also useful for those of us working on specific components > > with associated sample apps to have just that one app built constantly > > as we work. > > I don't understand this point: > ninja -C build && make -C examples/myexample > That works fine for building on the commandline, but does not work well for building as part of "build-on-save" inside an IDE, which is the biggest reason I want to keep support for building examples using meson. For doing development with a feature and associated sample app, being able to configure your build folder to rebuild a particular sample app as part of a main infrastructure rebuild is really useful - and works really quickly too, since incremental builds of C file changes happen really fast with meson. > > * Therefore, while it looks like the more logical part to drop is indeed > > the meson support for the examples, > > Yes, building the examples from the inside build system is strange, > and hide issues. > Sorry, while it may hide issues with the makefile because people weren't really aware that they were sticking around, it definitely helps pick up issues with C code changes as you are developing. > > we may struggle to implement clean > > building of the examples from meson using make, at least until meson 0.54 > > becomes our standard. Before that version, we don't have a > > "meson-uninstalled" folder with build-time package-config file to use as > > source of build flags for each example. > > We don't have to use meson at all. > No, we don't, so long as we don't miss out on the benefits we currently get from having it integrated. > > * One final idea I had and investigated in the past was whether we could at > > build or install time auto-generate the Makefile for each example from > > the meson.build file. Unfortunately, nothing came of that idea the first > > time I tried it, but it might still be worth looking at. Even if it works > > for 80-90% of cases, it means that we have a much smaller subset of > > examples where we need to test independently the make and meson builds. > > Hand-crafted Makefile is enough. They may be improved. > If we feel it is too hard, we can use another build system > in examples, like cmake. > > > So overall my assessment is that it needs a bit of investigation and > > prototyping to see what we can come up with. > > I think testing external build + removing build from internal meson > would be a good start to ensure quality of examples maintenance. > If we remove the meson build of the examples, I think we need equivalent functionality provided some other way. Calling make for examples from within meson may be good enough, so long as it's not too slow. > > On a semi-related note, it's perhaps a bigger problem that we cannot rely > > on test-meson-builds and CI infrastructure to prevent issues like this. > > We can. We just have to add all examples in test-meson-builds.sh. > Yep, definite +1 here. Once that is done, I'd then like to wait and see what future issues crop up before we start ripping out other bits. This is the first release where we have removed the make build system, so there are lots of teething issues all over the place, of which this is but one example! > > Surely this is what CI is there for - to help reduce the workload for > > maintainers. The fact that we are considering removing the meson build of > > examples because we cannot rely on CI is surely more worthy of a solution > > than trying to find a way to build examples with make from within meson? > > No the concern is to have all contributors work on the same > single build path. > Building all examples from test-meson-build should probably be sufficient here, I think. > > Perhaps we need to see about stricter measures from CI failure, e.g. > > anything failing travis build automatically gets marked as changes > > requested in patchwork, and the author gets an email informing them of > > such? > > When there is a failure, authors receive an email, > and maintainers can see a red flag. I think it's OK. > > The only issue was that this build path was not tested. > I think David is going to fix it by compiling all possible examples > with external make build from test-meson-builds.sh. > Seems like we are all aligned on the first step anyway. Let's see beyond that what needs to be done in 21.02. /Bruce