[Kstars-devel] strategies for expanding the size of catalogs

[James Bowlin]

On Mon October 15 2007, Jason Harris wrote:
> What do you think?  Is it a good idea to keep the brightest stars as
> a separate, non-regionalized catalog?  Will the progressive drawing
> work?

I think this is a wonderful idea Jason.  I agree with keeping the
brightest stars separate.  We are currently using a level-3 HTM
which has 512 region.  Keeping the dimmer stars in 512 different
batches makes sense but I have no idea what the optimal number
of regions is.  Another thing to consider is having redundant data
on disk in order to optimize memory usage.  In that scenario, the
brightest stars would be in one file (or batch) but they would
also be repeated in each of the regions so that when the user zooms
in to a region, we can dump the global set of brigtest stars.
We could even have several levels of regional files with a smaller
set of regional files that don't go to the dimmest levels.

I think we would get the best disk performance using flat binary
files sorted by brightness as you suggest.  Unfortunately binary
format will not be portable so we would probably want to generate
the binary files at install time or have separate data sets for
different architectures.  Also, I don't know if C++ is designed
to let us initialize objects by just loading in memory.  In C
and assembly, I am accustom to reading from disk directly into
essentially an array of structs.

Another idea would be to use a MySQL backend but I think we can
actually get better performance out of flat binary files.

Instead of (or in addition to) giving users a faintness magnitude
slider we could give them a slider for how much memory to use for 
holding stars.  The idea is that we can usually keep all of the
stars we display in RAM and only hit the disk when the user zooms
or pans (or the time changes significantly).  We should avoid
hitting the disk on every draw cycle if at all possible, maybe we
should just refuse to do it except in very special situations (such
as with automatic updating turned off).

A crude form of progressive drawing would be to first draw
the visible stars we have in memory then hit the disks and
trigger another draw after the new stars have been loaded.
I tried doing this when the faint magnitude was increased but
I had to give it up because the splash screen caused the main
window to gray out so the first draw with just the stars already
in memory was immediately erased.  We might want to solve this
existing problem before enlarging the catalog.

One potential show stopper would be proper motion.  We currently
re-index parts (or all of) our catalog whenever the user makes
a very large change in the time.  I don't know what to do about
this.

  
-- 
Peace, James