Another proposal for modernization of our infrastructure
jkt at kde.org
Sat Jan 31 13:01:22 GMT 2015
On Saturday, 31 January 2015 13:08:07 CEST, Inge Wallin wrote:
> Well, all of the above and more. Hosting, electricity, networking,
I'm including all of the above as "HW costs" in my proposal. We (KDE) do
not have our own datacenter after all.
> manual work as the number of physical machines increas
Due to the nature of build jobs which constitute a pretty bursty load,
renting VMs sounds like a cost-effective approach for our scale. I do not
expect that it would make financial sense for us to procure enough physical
HW to cover the peak demand -- hence the VMs. Renting VMs also enables
corporate sponsors to offer unused capacity for less money, or even for
Another benefit is that bootstrapping a new VM is about executing a single
command (and this is not hype; it's how all of the current build VMs in use
by Gerrit were launched).
> manual work as the complexity increases, and so on.
I would prefer if we stopped using phrases such as "increased complexity",
and switched to expressing a specific and measurable difference in costs.
The reason for this request is that "increased complexity" is almost always
true, yet people have no way of actually judging what it is going to mean.
For example, we have in-house expertise in Jenkins where Ben, Nicolas,
Scarlett and Marko understand the stack we use. Switching to anything else
will cost us some time to get the respective sysadmins up to speed to the
new tools. I do not question this.
However, chances are that at the same time adopting a new system could
provide significant savings in future. For example, if a new system allowed
easier updates to the build structure, it is reasonable to have a wiki page
with instructions and to let developers propose changes for their projects.
That way, sysadmin time could be freed for other tasks. So even though
there is some initial investment, the long-term operation can easily get
less demanding, and cheaper.
In fact, a better tooling could attract more people. "Hey, this is a nice
system which provides almost exactly what I need; can I improve it?" It
also presents an opportunity to clean some of the cruft accumulated over
years, thereby reducing the barrier of entry for new contributors and
improving the bus factor.
Finally, there is also that aspect of providing a better service to our
developers. Running infrastructure costs something, yet we do it. IMHO, we
should strive for offering the best service that we can afford with the
resources we have today, and have a contingency plan for future.
Right now, we have access to some nice CPUs for free. If it proves useful
and we get used to it and if the resources stop being free at some future
point and if we cannot obtain a free alternative from some other sponsor,
then we should take a look on how much money it would cost us on a
comercial basis at that point. If the price is too big, then the service is
*not* that useful for us after all, and we will live without it just fine.
If, however, we decide that the cost is worth the price and no free/cheaper
altenrative exists, then we should go and pay. There is no vendor lock-in,
and it is trivial to migrate to another provider. I just don't see why we
should be actively prevented from using resources which we have today just
because we might have to ask somewhere else in future. Turning off
pre-merge CI would get us back to the current level of resource utilization
That's what this proposal is all about, and please keep in mind that it's
something which is already deployed and tested. It isn't an
out-of-thin-blue proposal with uncertain cost of deployment.
> Everything can be automated in theory but in practice there is
> never a fully automatic system.
Yes, which is why it's important to looks at the architecture and to spell
out what our expectations of maintenance effort are. It might be reasonable
to e.g. compare them with the current effort which goes into keeping
Jenkins alive and modernizing it.
Keeping the current state has its very real cost, too.
With kind regards,
Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/
More information about the kde-core-devel