Re: [dpdk-dev] 19.11.4 patches review and test

[Christian Ehrhardt <christian.ehrhardt@canonical.com>]

On Wed, Mar 24, 2021 at 11:28 AM Pai G, Sunil <sunil.pai.g@intel.com> wrote:
>
> > -----Original Message-----
> > From: Christian Ehrhardt <christian.ehrhardt@canonical.com>
> > Sent: Wednesday, March 24, 2021 1:15 PM
> > To: Ilya Maximets <i.maximets@ovn.org>
> > Cc: Thomas Monjalon <thomas@monjalon.net>; Luca Boccassi
> > <bluca@debian.org>; Richardson, Bruce <bruce.richardson@intel.com>; Pai
> > G, Sunil <sunil.pai.g@intel.com>; Stokes, Ian <ian.stokes@intel.com>;
> > Govindharajan, Hariprasad <hariprasad.govindharajan@intel.com>;
> > stable@dpdk.org; dev <dev@dpdk.org>; James Page
> > <james.page@canonical.com>
> > Subject: Re: [dpdk-dev] 19.11.4 patches review and test
> >
> > On Tue, Mar 23, 2021 at 7:51 PM Ilya Maximets <i.maximets@ovn.org>
> > wrote:
> > >
> > > On 3/23/21 7:17 PM, Thomas Monjalon wrote:
> > > > 22/03/2021 15:27, Christian Ehrhardt:
> > > >> On Mon, Mar 22, 2021 at 1:25 PM Thomas Monjalon
> > <thomas@monjalon.net> wrote:
> > > >>> 22/03/2021 12:59, Luca Boccassi:
> > > >>>> On Mon, 2021-03-22 at 11:41 +0000, Bruce Richardson wrote:
> > > >>>>> On Mon, Mar 22, 2021 at 10:49:54AM +0100, Christian Ehrhardt
> > wrote:
> > > >>>>>> On Thu, Mar 18, 2021 at 7:25 PM Pai G, Sunil
> > <sunil.pai.g@intel.com> wrote:
> > > >>>>>>> Hi Christian, Ilya
> > > >>>>>>> From: Ilya Maximets <i.maximets@ovn.org>
> > > >>>>>>>> On 3/18/21 2:36 PM, Pai G, Sunil wrote:
> > > >>>>>>>>> Hey Christian,
> > > >>>>>>>>>
> > > >>>>>>>>> <snipped>
> > > >>>>>>>>>
> > > >>>>>>>>>> back  in 19.11.4 these DPDK changes were not picked up as
> > > >>>>>>>>>> they have broken builds as discussed here.
> > > >>>>>>>>>> Later on the communication was that all this works fine now
> > > >>>>>>>>>> and thereby Luca has "reverted the reverts" in 19.11.6 [1].
> > > >>>>>>>>>>
> > > >>>>>>>>>> But today we were made aware that still no OVS 2.13 builds
> > > >>>>>>>>>> against a DPDK that has those changes.
> > > >>>>>>>>>> Not 2.13.1 as we have it in Ubuntu nor (if it needs some
> > > >>>>>>>>>> OVS changes
> > > >>>>>>>>>> backported) the recent 2.13.3 does build.
> > > >>>>>>>>>> They still fail with the very same issue I reported [2] back
> > then.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Unfortunately I have just released 19.11.7 so I can't
> > > >>>>>>>>>> revert them there - but OTOH reverting and counter
> > > >>>>>>>>>> reverting every other release seems wrong anyway.
> > > >>>>>>>>
> > > >>>>>>>> It is wrong indeed, but the main question here is why these
> > > >>>>>>>> patches was backported to stable release in a first place?
> > > >>>>>>>>
> > > >>>>>>>> Looking at these patches, they are not actual bug fixes but
> > > >>>>>>>> more like "nice to have" features that additionally breaks the
> > way application links with DPDK.
> > > >>>>>>>> Stuff like that should not be acceptable to the stable
> > > >>>>>>>> release without a strong justification or, at least, testing with
> > actual applications.
> > > >>>>>>
> > > >>>>>> I agree, but TBH IIRC these changes were initially by OVS
> > > >>>>>> people :-) One could chase down the old talks between Luca and
> > > >>>>>> the requesters, but I don't think that gains us that much.
> > > >>>>>>
> > > >>>>>>>> Since we already have a revert of revert, revert of revert of
> > > >>>>>>>> revert doesn't seem so bad.
> > > >>>>>>
> > > >>>>>> As long as we don't extend this series, yeah
> > > >>>>>>
> > > >>>>>>>>>> I wanted to ask if there is a set of patches that OVS would
> > > >>>>>>>>>> need to backport to 2.13.x to make this work?
> > > >>>>>>>>>> If they could be identified and prepared Distros could use
> > > >>>>>>>>>> them on
> > > >>>>>>>>>> 2.13.3 asap and 2.13.4 could officially release them for OVS
> > later on.
> > > >>>>>>>>>>
> > > >>>>>>>>>> But for that we'd need a hint which OVS changes that would
> > need to be.
> > > >>>>>>>>>> All I know atm is from the testing reports on DPDK it seems
> > > >>>>>>>>>> that OVS
> > > >>>>>>>>>> 2.14.3 and 2.15 are happy with the new DPDK code.
> > > >>>>>>>>>> Do you have pointers on what 2.13.3 would need to get
> > > >>>>>>>>>> backported to work again in regard to this build issue.
> > > >>>>>>>>>
> > > >>>>>>>>> You would need to use partial contents from patch :
> > > >>>>>>>>>
> > http://patchwork.ozlabs.org/project/openvswitch/patch/160814
> > > >>>>>>>>> 2365-
> > > >>>>>>>> 26215
> > > >>>>>>>>> -1-git-send-email-ian.stokes@intel.com/
> > > >>>>>>>>>
> > > >>>>>>>>> If you'd like me to send patches which would work with 2.13,
> > > >>>>>>>>> 2.14, I'm ok with that too.[keeping only those parts from
> > > >>>>>>>>> patch which fixes the issue
> > > >>>>>>>> you see.] But we must ensure it doesn’t cause problems for
> > OVS too.
> > > >>>>>>>>> Your thoughts Ilya ?
> > > >>>>>>>>
> > > >>>>>>>> We had more fixes on top of this particular patch and I'd
> > > >>>>>>>> like to not cherry- pick and re-check all of this again.
> > > >>>>>>>
> > > >>>>>>> I agree, we had more fixes on top of this. It would be risky to
> > cherry-pick.
> > > >>>>>>> So it might be a better option to revert.
> > > >>>>>>
> > > >>>>>> I agree, as far as I assessed the situation it would mean the
> > > >>>>>> revert of the following list.
> > > >>>>>> And since that is a lot of "reverts" in the string, to be clear
> > > >>>>>> it means that those original changes would not be present
> > anymore in 19.11.x.
> > > >>>>>>
> > > >>>>>> f49248a990 Revert "Revert "build/pkg-config: prevent
> > overlinking""
> > > >>>>>> 39586a4cf0 Revert "Revert "build/pkg-config: improve static linking
> > flags""
> > > >>>>>> 906e935a1f Revert "Revert "build/pkg-config: output drivers
> > > >>>>>> first for static build""
> > > >>>>>> deebf95239 Revert "Revert "build/pkg-config: move pkg-config file
> > creation""
> > > >>>>>> a3bd9a34bf Revert "Revert "build: always link whole DPDK static
> > libraries""
> > > >>>>>> d4bc124438 Revert "Revert "devtools: test static linkage with pkg-
> > config""
> > > >>>>>>
> > > >>>>>> But to avoid going back&forth I'd prefer to have a signed-off
> > > >>>>>> on that approach from:
> > > >>>>>> - Luca (for 19.11.6 which has added the changes)
> > > >>>>>> - Bruce (for being involved in the old&new case in general)
> > > >>>>>> - Thomas (for general master maintainer thoughts)
> > > >>>>>>
> > > >>>>>
> > > >>>>> If this is what is needed to ensure OVS can continue to use this
> > > >>>>> release series, then I am absolutely fine with it.
> > > >>>>
> > > >>>> This was requested by OVS, so if they don't need it anymore it's
> > > >>>> fine by me as well
> > > >>>
> > > >>> I am not sure to understand the full story, but I am a bit worried
> > > >>> that our release is dictated by a single "user" (project using
> > > >>> DPDK).
> > > >>
> > > >> Sure, fair to ask for more detail :-)
> > > >>
> > > >>> Please do you have links of discussion history?
> > > >>
> > > >> I ordered the events by time and added links to those occasions
> > > >> that I could find:
> > > >>
> > > >> July 2020            - Initial request by OVS - *1
> > > >> July 2020            - Initial queuing     -
> > > >> http://mails.dpdk.org/archives/stable/2020-July/024248.html
> > > >> September 2020 - Issues identified; changes reverted    -
> > > >> http://mails.dpdk.org/archives/stable/2020-September/024796.html
> > > >> October 2020      - Re-applying early in 19.11.6 cycle    - *1
> > > >> November 2020  - Tests didn't spot it with 19.11.6 as OVS 2.14.x (not
> > > >> the 2.13 LTS) was tested    -
> > > >> https://doc.dpdk.org/guides-19.11/rel_notes/release_19_11.html#id16
> > > >> March 2021         - Same issue re-found in >=19.11.6    -
> > > >> http://mails.dpdk.org/archives/stable/2021-March/029418.html
> > > >>
> > > >> *1 - Luca and I looked for logs, there are no links that I'd know
> > > >> of and Luca said it might have come up as a request during a meeting.
> > > >
> > > > First, I agree to revert the changes again if it causes a regression.
> > > > Second, do we know the root cause of the issue?
> > > > Is it a problem with the version of pkg-config?
> > > > Is it OK with DPDK 20.11?
> > > >
> > >
> > > I'd like to also ask someone to test build of both OVS 2.13 and OVS
> > > 2.14 with these changes and with these changes reverted.
> >
> > I've test built a few of those already.
> > - 19.11.4 (before the patches were applied)
> >   - OVS 2.13.1 worked
> > - 19.11.6/19.11.7 (patches not yet reverted)
> >   - OVS 2.13.1 fails
> >   - OVS 2.13.3 fails
> > - 19.11.7 patches reverted
> >   - OVS 2.13.3 works
> >
> > I'd also be happy to hear about OVS 2.14 test builds, so yeah if you could do
> > so  @Sunil that would be great.
>
> Tested 19.11 series with OVS 2.14 and observations are like your's Christian.
> 19.11.4 and 19.11.7 -with patches reverted works fine, 19.11.6/7(patches not yet reverted) cause linking errors.

Thank you, with those tests and all the ack's in I'll push this to the
19.11 branch on the dpdk-stable repo.

> > For the code, I've not yet pushed it to "real dpdk-stable" until we are sure
> > about it, but already to:
> >   https://github.com/cpaelzer/dpdk-stable-queue/tree/19.11
> > If you happen to build on Ubuntu there is a 19.11.7 + reverts already available
> > here
> >   https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3690/
> >
> > > Sunil, could you do that?
> > >
> > > > About the process, I see multiple issues:
> > > >
> > > > 1/ Some patches were backported for OVS only, but it could break
> > > > other applications.
> >
> > As we found it even breaks (older) OVS, but importantly the OVS LTS which
> > has the highest chance to be in use together with DPDK 19.11 in many places
> > :-/
> >
> > > > 2/ It is not clear whether the patches were really needed in 19.11.
> > > >
> > > > 3/ There is no trace of backport requests in the mailing list.
> > > >
> > > > So I feel we should be stricter on the reasons for a backport.
> > > > Note: I am not blaming anyone. Everybody tries to do the best.
> > > > I believe sharing requests and discussions on the mailing list could
> > > > help in the decision process.
> >
> > Agreed
> >
> > > +1
> > >
> > > >
> > > > Thanks for all the work.
> >
> >
> >
> > --
> > Christian Ehrhardt
> > Staff Engineer, Ubuntu Server
> > Canonical Ltd



-- 
Christian Ehrhardt
Staff Engineer, Ubuntu Server
Canonical Ltd