very large data files

Jason Harris kstars at 30doradus.org
Thu Jul 10 05:25:16 BST 2008


Hello,

Our GSoC student is working on making KStars display 1 million stars,
without adversely affecting the program's responsiveness.  He's doing
so well that we're already making plans to try 10 million, 100
million, and even (when we're feeling insane) 1 billion stars.

The problem is that the binary data file for even 1 million stars is
already 74 MB.  This is larger than the entire kstars directory in
svn, so it's clearly not something we want to store in the kdeedu
module.  I'm wondering if it's even too big to make available through
GHNS (although I guess it would be okay as long as we note the large
file size in the description text).

You can guess what the real problem is: the 10 million star data file
is going to be about 750 MB, and 100 million stars will be 7.5 GB.
How are we going to distribute these files?  Is KDE billed per-bit for
transfers from kde.org?

You'll note I didn't even mention the file containing 1e9 stars.  The
US Naval Observatory publishes a catalog with 1e9 stars called NOMAD
(which our large catalogs are based on), and to obtain it you have to
send them a USB hard drive with a capacity of at least 100 GB.  We'd
probably have to use a similar method.

thanks for any advice,
Jason




More information about the kde-core-devel mailing list