[Kstars-devel] strategies for expanding the size of catalogs

[Jan Kotek]

> I think we would get the best disk performance using flat binary
> files sorted by brightness as you suggest.  Unfortunately binary
> format will not be portable so we would probably want to generate
> the binary files at install time or have separate data sets for
> different architectures.  Also, I don't know if C++ is designed
> to let us initialize objects by just loading in memory.  In C
> and assembly, I am accustom to reading from disk directly into
> essentially an array of structs.
>
> Another idea would be to use a MySQL backend but I think we can
> actually get better performance out of flat binary files.
>

I was experimenting with MySQL and some star catalogs. For N > 10^7
flat file is better. Problem is with index which must fit to memory,
otherwise DB starts swaping.

But I would recommend embedded MySQL anyway. It solves all problems
with caching and data format. Also MyISAM tables can be compressed
(and readonly) and distributed without index. It is very space
efficient. Index can be created during installation (depency on
mysql-tools).

My program is using SQL database and HTM index. It already can read
about 60 catalogs and I am going to convert all of them to database
tables in near future. Maybe you can use my tables, it  saves you lot
of work.

Jan
http://kotek.net/opencoeli