General Infrastructure Maintainability: API and EBN

[Albert Astals Cid]

El diumenge, 10 de novembre de 2019, a les 1:48:10 CET, Ben Cooksley va escriure:
> On Sun, Nov 10, 2019 at 11:20 AM Albert Astals Cid <aacid at kde.org> wrote:
> >
> > El dissabte, 9 de novembre de 2019, a les 22:03:37 CET, Ben Cooksley va escriure:
> > > On Sun, Nov 10, 2019 at 9:29 AM Alexander Potashev <aspotashev at gmail.com> wrote:
> > > >
> > > > сб, 9 нояб. 2019 г. в 21:50, Ben Cooksley <bcooksley at kde.org>:
> > > > > > > Over the past number of years one of the tasks the Sysadmin team has
> > > > > > > worked on has been improving the overall maintainability of our
> > > > > > > systems, with a significant number of specialised cronjobs, exceptions
> > > > > > > and hidden linkages being eliminated.
> > > > > > >
> > > > > > > That is with one great exception: api.kde.org and ebn.kde.org.
> > > > > > >
> > > > > > > Both of these are suffering from an extreme amount of digital bitrot
> > > > > > > and special casing and in general are now in a condition where I
> > > > > > > cannot say for certain whether it would be possible to replicate the
> > > > > > > setup on a new system without us experiencing some degree of breakage
> > > > > > > (some of which we may not discover until weeks/months afterwards).
> > > > > > >
> > > > > > > In addition, the current setup relies on an old-fashioned overnight
> > > > > > > reprocessing of all repositories, which is inefficient and resource
> > > > > > > expensive. A more modern approach would have the various projects api
> > > > > > > documentation generated on a delayed cycle from relevant branches as
> > > > > > > part of something like a CI job (but not part of the actual CI
> > > > > > > workflow itself).
> > > >
> > > > Hi Ben,
> > > >
> > > > I can't discuss this topic before we understand what exactly is wrong
> > > > with api.kde.org and ebn.kde.org and why they are hard to manage.
> > > > Could you please describe the current situation (where to find source
> > > > code, how many servers, etc) in new Phabricator tasks like you did for
> > > > identity.kde.org in https://phabricator.kde.org/T8449 ?
> > > >
> > > > P.S.  Complicated legacy systems can be easy to replicate once you
> > > > automate their deployment by using Docker containers and/or
> > > > configuration management software like Ansible.
> > >
> > > The problem isn't just the complicated legacy nature of them, but also
> > > how fragile and impossible to maintain they are.
> > >
> > > There are currently to my knowledge the following ways of generating
> > > documentation within the system as it currently stands:
> > > - Legacy KDE 4 era generation tooling, still in use for large parts of
> > > KF5 based applications (which needs periodic fixes, see
> > > https://cgit.kde.org/kdelibs.git/commit/?id=12a8bb503351b98869f722ba932f822e3c495883)
> > > - DoxyQML, which is in part reliant on the above KDE 4 era generation tooling
> > > - Specialist handling for 'cmakedocs' and 'mkdocs' based projects
> > > - New era KApiDox tooling
> > >
> > > On top of all of this, the entire process can only be run as one
> > > single monolithic piece, which makes debugging and investigating
> > > faults difficult.
> > >
> > > To use an example of this, back in February 2018 we received a request
> > > (Sysadmin ticket) from someone reporting that their project's API
> > > documentation wasn't being generated.
> > > Despite investigation by Allen and others, we were unable to resolve
> > > the issue, and recommended that the project in question be switched
> > > from the KDE 4 era tooling to the newer KApiDox tooling, as it wasn't
> > > possible to identify the fault.
> > >
> > > Earlier this year there was a fault with API generation in general
> > > which broke a number of projects due to encoding of filenames/folders
> > > (fixed in https://cgit.kde.org/websites/quality-kde-org.git/commit/?id=f71c68bc80cce68aa3bcc6a51626c8f22b7c73c3).
> >
> > I volunteer to fix it.
> 
> Hi Albert,
> 
> Thanks for the offer of help.
> 
> Please note though that this is much more than just fixing what we
> currently have.
> It is re-engineering what we currently have to ensure that it is
> maintainable, and meets the broader needs of the community in the long
> term.

Yes, i understand that.

> 
> What we have currently doesn't really meet those standards - and as I
> mentioned earlier, is in some cases impossible to debug and therefore
> fix as everything appears to be working properly but ends up not
> working.
> 
> It would also be nice if we could use this as an opportunity for
> someone not yet deeply involved to get involved more closely.

That's kind of impossible though, asking a total newbie to take care of re-engineering a "old system" is almost a recipe for 0% hands raised.

Cheers,
  Albert

> 
> >
> > Cheers,
> >   Albert
> >
> 
> Cheers,
> Ben
> 
> > >
> > > >
> > > > --
> > > > Alexander Potashev
> > >
> > > Cheers,
> > > Ben
> > >
> >
> >
> >
> >
>