Proposal to improve support for ISO 3166 Country Codes in KLocale

[John Layt]

Apologies for the delay in replying, I was at FOSDEM and my laptop was getting 
used as the booth demo unit :-)

On Saturday 06 Feb 2010 05:35:24 Albert Astals Cid wrote:
> > A Dijous, 4 de febrer de 2010, John Layt va escriure:
> > My big question is over translations.  The XML file contains subdivision
> > names in their local version, so even en_US would require translating. 
> > I assume we would be able to use the provided translations, or would we
> > have to do our own?
> 
> Depends on how translations are provided and what you want to do with them.
> Please elaborate.

> My question is how John wants to use that XML files and if it's just XML
> files using the lang="foo" attribute or it's XML files + .po files for
> translation or what.

The iso-codes project appears to use standard po files that get installed to 
/usr/share/locale/<lang>/LC_MESSAGES/.  The xml file only has subdivision 
names in a local form (but in Latin script), not the English name, so every 
language including en_US will need to translate the xml name every time.  
There is no lang attribute in the xml file.

Here's some edited extracts from the file just to illustrate:

<!-- Belgium -->
<iso_3166_country code="BE">
<iso_3166_subset type="Province">
	<iso_3166_2_entry
		code="BE-VOV"	name="Oost-Vlaanderen" />
</iso_3166_subset>
</iso_3166_country>

<!-- Bahrain -->
<iso_3166_country code="BH">
<iso_3166_subset type="Governorate">
	<iso_3166_2_entry
		code="BH-13"	name="Al Manāmah (Al ‘Āşimah)" />
</iso_3166_subset>
</iso_3166_country>

<!-- Spain -->
<iso_3166_country code="ES">
<iso_3166_subset type="Autonomous communities">
	<iso_3166_2_entry
		code="ES CT"	name="Cataluña" />
</iso_3166_subset>
<iso_3166_subset type="Province">
	<iso_3166_2_entry
		code="ES-B"	name="Barcelona"	parent="CT" />
</iso_3166_subset>
<iso_3166_subset type="Autonomous city">
	<iso_3166_2_entry
		code="ES-CE"	name="Ceuta" />
</iso_3166_subset>
</iso_3166_country>

See http://git.debian.org/?p=iso-codes/iso-
codes.git;a=blob;f=iso_3166_2/iso_3166_2.xml;hb=HEAD

If integrated into KLocale, then KLocale would provide methods like:

    QStringList countrySubdivisionList( const QString &countryCode )

		Load the xml file and return the subdivision codes for the country

    QString countrySubdivisionName( const QString &subdivisionCode )

		Load the xml file, read the matching name attribute, translate via po

There could be caching, whatever proves more efficient overall.  We could 
provide a class to encapsulate the data structure including subdivision type 
and the hierarchy if required.

At the moment I have only one definite use in KDE, that is with KHolidays 
where we want to split and tag holiday files by country subdivisions, i.e. for 
the USA to allow the user to display state holidays for California but not 
Texas or any other states.  It makes sense to use the ISO codes for this 
tagging internally rather than the full English name.  The translations would 
be used in System Settings to let the user choose their subdivision as a 
global default, and in KOrganizer and the Plasma Calendar where we would list 
the available files for the user to select from including the subdivision 
name.

Short term this name translation can obviously just be done on a per holiday 
file basis in a cpp somewhere, but medium term the numbers will hopefully grow 
to the point where this becomes inconvenient, especially if we add GHNS to 
download new files.

Long term we also intend to agree with fd.o on a shared xml holiday file 
format and storage location, so at that stage translations and their 
implementation method would become a shared issue.  Using an existing 
translation source now could have advantages for this, but that's some way 
off.

Obviously a single use case is not enough for inclusion in kdelibs, hence the 
suggestion KGeography and Marble may also be interested.  I'm sure there's 
other potential uses like web-services, default timezones, etc.

KGeography already has many of the subdivisions included (but not using the 
ISO codes) and names translated so might not be interested, but all those 
existing translations could be utilised by KHolidays.

I'd really like to hear what the Marble guys think or have planned in this 
area.

If there's not enough interest, I'll just use the codes we need in KHolidays 
and translate what we need in cpp until it gets too big or we need more 
flexibility.

[Aside: we need to sort out flags too sometime, kdebase, kgeography, and 
marble all have their own sets.  The Marble svg flags from Wikipedia look the 
best bet, a move of marble widget to kdesupport could solve both nicely :-)]

> >  In either event, not all KDE supported languages are
> >  included in iso- codes, would we be able to mix their translations and
> >  our own?  Or would we prefer to do our own for all languages?
> 
> If we are going to use an external dependency, that dependency must accept
> translations, we have own translations for some not friendly projects like
> Qt or xscreensaver but i would object to having a new dependency that does
> not accept translations from our translators.

OK, so we'd contribute directly to their translations and then link in their 
po files via gettext.  I assume we work like this with other projects?  Do you 
have procedures and standards you like to see in place in the project to judge 
how good/open they are?

> You can't demand anything since it's an external dependency so we don't
> have a control over its translations (if you follow my suggestion, that's
> it :D)

I was meaning demand of the KDE translators if we went the internal route, I 
know how much work you guys already have, demanding even more just because I 
want 4200 more strings in kdelibs would be unreasonable of me.  I'm sure if I 
suddenly dumped that many into kdelibs a lynch mob would soon be after me :-D

If you were to work with upstream to add the translations into their files 
then the files would lie outside KDE so wouldn't affect your rules for 
language inclusion, even if it meant stuff showed up untranslated?

> > Besides KLocale providing access methods, System Settings would allow
> > selection of the users default subdivision.  We could also in theory
> > provide locale files for any country subdivision with localisation
> > standards that differ from the rest of the country.
> 
> No, upstream them, if you are going to use a package, use it, don't start
> shipping custom files for it just because you can.

Do you mean the translations or the locale files?  I'd agree with the 
translation files, but when I say locale files I mean the ones in 
kdebase/runtime/l10n that define number and date formats and preferred 
languages and so on.  Does Quebec use different formats to the English 
speaking rest of Canada?  Does Catalonia use different formats to the rest of 
Spain?  I don't know yet.  If so we could have separate files with the correct 
default settings for those regions.

> And if the package is orphaned or the maintainer sucks, don't use it.

The project is at least 5 years old and still appears to be very active with 
very regular commits, and the more projects that use it the more likely that 
it will remain active.  I intend to contact them if we decide it looks a good 
option to seek some more background and reassurances.  Worst comes to worst, 
we can always just pull what we need into kdelibs and maintain it ourselves, 
but I'd rather not.

Cheers!

John.