Sphinx Application Documentation - Image duplication

Sun Jan 22 19:59:26 GMT 2023

Hi Eugen,

at least in the Kdenlive docs we do not use translated images at all. Also the figure_language_name is to define the pattern from where translated images are taken not where they are put to. And we always have the problem that Sphinx uses separate build dir for every language so unfortunately this does not help at all for the problem Ben exposed.

Cheers,
Julius

22.01.2023 20:46:27 Eugen Mohr <eugen.mohr at gmx.net>:

> Hi Julius
> 
> Maybe we should do the opposite of this: https://github.com/sphinx-doc/sphinx/issues/9758 and this: https://stackoverflow.com/questions/39008064/how-to-customize-sphinx-doc-images-folder
> 
> The language-specific figures/images are set here: https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-figure_language_filename -> can we switch this off / do the opposite?
> 
> Eugen
> 
> Am 22.01.2023 um 20:23 schrieb Julius Künzel:
>> Hi Ben,
>> 
>> yes, that seems to be the most proper solution for me, and I think this should work
>> 
>> Cheers,
>> Julius
>> 
>> 22. Januar 2023 um 20:02, "Ben Cooksley" <bcooksley at kde.org[bcooksley at kde.org?to=%22Ben%20Cooksley%22%20%3Cbcooksley%40kde.org%3E]> schrieb:
>> 
>> On Mon, Jan 23, 2023 at 7:51 AM Julius Künzel <jk.kdedev at smartlab.uber.space> wrote:
>> Hi Ben, hi all,
>> 
>> Hi Julius,
>>  
>> 
>> I did a little research about this recently and unfortunately it seems to me as if there is not really a solution on the Sphinx side. One need to have separate build dirs for every language and it copies all static files (css, js, images,..) to every build dir. That's just how it works :-/ (Correct me in case anyone knows I am wrong).
>> However we can of course try to solve this on our and and make our deploy tools smart in a way that they keep only one version of each image file and replace the others with symlinks.
>> It should be more or less easy to detect images that are translated since they follow the pattern *filename.de.png where "de" is the language code, so this image would be special for German, while for all other languages filename.png is used.*
>> 
>> 
>> I had a very strong feeling that would be the case (very much seems that Sphinx actually doesn't have proper i18n/l10n support and it's been hacked in / bolted on later).
>> 
>> My initial thinking on a quick and (somewhat) dirty solution to this had been to merge all of the image files into a single folder at top level and then symlink that from each language.
>> Knowing that translated images actually have a separate filename convention indicates that this might just be crazy enough to work.
>> 
>> Thoughts?
>>  
>> 
>> I hope that helps so far. I might be able to look into this, but probably not very soon so if anybody else can work on this I am more than happy.
>> 
>> Cheers,
>> Julius
>> 
>> Regards,
>> Ben
>>  
>> 
>> 15. Januar 2023 um 07:45, "Ben Cooksley" <bcooksley at kde.org[bcooksley at kde.org?to=%22Ben%20Cooksley%22%20%3Cbcooksley%40kde.org%3E]> schrieb:
>> 
>> Hi all,
>> 
>> For some time now it has been known to me that the system for generating application documentation websites using Sphinx with l10n support has had issues with duplicating data - particularly images.
>> 
>> That leads to the following outcome, where aside from sites that we expect to be quite large (like www.kde.org[http://www.kde.org/] and api.kde.org[http://api.kde.org/]) all of the application documentation sites are quite big as well:
>> 
>> root at nicoda /srv/www # du -h --max-depth=1 ./generated/ | grep G
>> 2.3G    ./generated/cutehmi.kde.org[http://cutehmi.kde.org/]
>> 3.7G    ./generated/docs.digikam.org[http://docs.digikam.org/]
>> 2.4G    ./generated/api.kde.org[http://api.kde.org/]
>> 2.3G    ./generated/docs.krita.org[http://docs.krita.org/]
>> 1.4G    ./generated/www.kde.org[http://www.kde.org/]
>> 7.9G    ./generated/docs.kdenlive.org[http://docs.kdenlive.org/]
>> 29G     ./generated/
>> 
>> This stands in comparison to the Docbook documentation site for all other KDE applications:
>> 
>> root at nicoda /srv/www # du -h --max-depth=1 . | grep G
>> 29G     ./generated
>> 16G     ./api.kde.org-legacy
>> 6.0G    ./docs.kde.org[http://docs.kde.org/]
>> 51G     .
>> 
>> It would be nice if we could please look into some fixes for this, as it looks like Sphinx is duplicating the images - once for every language - when that isn't necessary.
>> I could understand if the screenshots were updated as part of the translation, but it looks like they're not in the majority of cases - below being just a sample:
>> 
>> root at nicoda /srv/www/generated/docs.krita.org[http://docs.krita.org/] # sha256sum zh_CN/_images/Krita_cpb_mixing.gif
>> 12eb4cbad29a5a6486d3438dabb888a0aa0b9579e55b3be2f3c1d6e1d76fc1d7  zh_CN/_images/Krita_cpb_mixing.gif
>> root at nicoda /srv/www/generated/docs.krita.org[http://docs.krita.org/] # sha256sum en/_images/Krita_cpb_mixing.gif
>> 12eb4cbad29a5a6486d3438dabb888a0aa0b9579e55b3be2f3c1d6e1d76fc1d7  en/_images/Krita_cpb_mixing.gif
>> 
>> While this isn't a massive issue right now, it is a future scalability issue as for Krita at least each language costs 178MB or so, while for Digikam that sits at 415MB per language and Kdenlive is 392MB.
>> 
>> Many thanks,
>> Ben
>> 
>> 
>> Julius Künzel
>> Volunteer KDE Developer, mainly hacking Kdenlive
>> KDE GitLab: https://my.kde.org/user/jlskuz/
>> Matrix: @jlskuz:kde.org[http://kde.org/]
>> 
>> 
>> 
>> Julius Künzel
>> Volunteer KDE Developer, mainly hacking Kdenlive
>> KDE GitLab: https://my.kde.org/user/jlskuz/
>> Matrix: @jlskuz:kde.org
>> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kimageshop/attachments/20230122/fc752898/attachment-0001.htm>