very large data files

[Jason Harris]

Hi Robert,

> I haven't looked at how you store the data but will you run into
> performance problems loading such a large data file into
> memory in any case?

This is one of the clever bits: until now we did indeed load all stars
into memory, but no longer.  Now the sky is divided into N subregions
(currently 512), and only the stars in on-screen regions are loaded
into memory.  We can seek/read the data from the file quickly because
it's in binary format and cleverly organized.

> Does anyone provide a streaming service which allows you to
> obtain only the subset of the data which is of interest? - Like
> Google Earth but for stars.

Yes, this is an option now that we have on-demand loading of stars.
Just stream the required data chunks over the network instead of
reading them from disk.  Latency would be an issue (I'd rather not
have to wait for stars to appear when I move to a new position like
you must do in GE).

regards,
Jason