[kde-doc-english] No docs on d.k.o for stable

[Burkhard Lück]

Am Freitag, 25. Oktober 2013, 20:18:49 schrieb Luigi Toscano:
> Albert Astals Cid wrote:
> > El Divendres, 25 d'octubre de 2013, a les 18:34:16, Yuri Chornoivan va
> > escriure:
> >> Hi,
> >> 
> >> Just FYI, due to network error there will be no docs on d.k.o for stable
> >> this week.
> > 
> > That's something that should be fixed so the script fails gracefully if
> > the updating somehow doesn't work.
> 
> The script on library.kde.org is an simple rsync call:
> 07 02 * * fri /usr/bin/rsync -azvC --delete /home/docs/website/
> docs at spider.kde.org:/srv/www/docs
> 
The rsync command did not fail, populate_documentation.sh for stable in 
generator script did nor work properly due to network errors.
> 
> Maybe it should upload the content to a separate directory and, only if the
> copy is complete, another script (on docs.k.o?) should move the files under
> the correct directory, so that a complete version is always available.
> 
> > Who's in charge of that script/server?
> 
> library.kde.org? Burkhard and me have root access, Yuri can access the
> "docs" user (the rsync line above is in the crontab of the "docs" user).
> 
We have so far several issues with d.k.o updates:

1) To switch stable release version you need to change release number in two 
different files (see e.g. 
http://websvn.kde.org/?view=revision&sortby=date&revision=1331259 ), but one 
change was forgotten for the switch of stable to 4.12, the generator script 
still has stable_branch="4.10"

2) Actually we remove all docbooks in stable/trunk (generator script line 26) 
for two reasons:
* Afaik populate_documentation.sh does not delete an obsolete directory
* Switching stable fails (see comment #3 in 
https://bugs.kde.org/show_bug.cgi?id=308104 )
But it would be sufficient to rm the docs only when switching stable, that would 
prevent the situation we have now.
 
3) Crontab for docs user on library.kde.org has 
"MAILTO=sysadmin at kde.org,lueck at hube-lueck.de"
This week I got two mails from 	Cron Daemon <root at library.kde.org>:
a) Cron <docs at library> cd /home/docs/docs && svn up
    "At revision 1367432."
   ->  it is not possible to detect that "svn up" failed partially
b) Cron <docs at library> /usr/bin/rsync -azvC --delete /home/docs/website/ 
docs at spider.kde.org:/srv/www/docs
   "sending incremental file list
   ./
   generated.inc.php
   [snip]
   sent 162033222 bytes  received 8728263 bytes  531139.92 bytes/sec
   total size is 1034905291  speedup is 6.06
   -> A *BIG* mail (10,7 MB)

Some ideas to solve 1) - 3)

* One place to change the release version, e.g a file "d.k.o-release-numbers" 
with "stable=4.11" and "development=4.12.x" in www/areas/docs/.

The generator script and index.php pull in these release numbers.

The release number for stable can be used on liberty.kde.org to check if the 
old docs need to be removed if the stable release number changes.

In pseudo code this would be something like this at the beginning of the 
generator script:

read stable from a file called "stable-release-number" with "stable=4.x"
if no file "stable-release-number" or stable != stable in "d.k.o-release-
numbers"
   delete docbooks (rm -rf $websiteDir)
   write stable from "d.k.o-release-numbers" to "stable-release-number"
endif

* remove MAILTO from Crontab

* At the end of generator script mail logs/generator.log to this list.
generator.log shows if updating the docbooks failed

* New script for rsync run in Crontab
Content of this script e.g.:
read line numbers from "rsync2spider.log" (=log from last run)
run "/usr/bin/rsync -azvC --delete /home/docs/website/ 
docs at spider.kde.org:/srv/www/docs" with output to "rsync2spider.log"
read new line numbers from "rsync2spider.log"(=log from current run)
mail both line numbers to this list

A big difference in both line numbers should indicate an error with syncing

What do you think?

-- 
Burkhard Lück