[kstars-devel] analysis of memory consumption

[Heiko Evermann]

Hi everyone,

I have analyzed the memory behavior of kstars:
Our observation from the process stats is that we need about 500 bytes 
per star
when we force loading of all stars.

How big are the objects that we are dealing with?
This can be found out by using sizeof:
bool KStarsData::readStarData( void ) {
 kdDebug() << "size of QString " << sizeof (QString);
 kdDebug() << "size of Star Object " << sizeof (StarObject);
 kdDebug() << "size of dms " << sizeof (dms);
 kdDebug() << "size of QStringList " << sizeof (QStringList);
 kdDebug() << "size of double " << sizeof (double);
 kdDebug() << "size of bool " << sizeof (bool);
 kdDebug() << "size of SkyPoint " << sizeof (SkyPoint);
 kdDebug() << "size of SkyObject " << sizeof (SkyObject);

The (commented) result is this:

double:
size=8 bytes

bool:
size=1 byte

QString:
size=4 bytes (for empty string, plus dynamically allocated content, 
which might be shared??)

QStringList
size = 4 bytes (for empty list, plus dynamically allocated content)

dms:
size=36 bytes (via sizeof)
details:
double  D              = 8 bytes
mutable double Radians = 8 bytes
mutable double Sin     = 8 bytes
mutable double Cos     = 8 bytes
mutable bool scDirty   = 1 byte
mutable bool rDirty    = 1 byte
sum: 34 bytes
2 bytes not accounted for. Rest might be padding to get a multiple of a 
dword as size

SkyPoint:
size=292 Bytes (via sizeof)
dms RA0     = 36 bytes
dms Dec0    = 36 bytes
dms RA      = 36 bytes
dms Dec     = 36 bytes
dms Alt     = 36 bytes
dms Az      = 36 bytes
dms galLat  = 36 bytes
dms galLong = 36 bytes
sum: 8*36 bytes = 288
plus 4 bytes for the pointer to vtable
all bytes accounted for

SkyObject
size=332 bytes (via sizeof)
is a SkyPoint  => 292 bytes, vtable pointer already included
unsigned char Type  1 byte
float Magnitude     4 bytes
QString Name        4 bytes (+ variable content)
QString Name2       4 bytes (+ variable content)
QString LongName    4 bytes (+ variable content)

QStringList ImageList  4 bytes
QStringList ImageTitle 4 bytes
QStringList InfoList   4 bytes
QStringList InfoTitle  4 bytes
QString     userLog    4 bytes
sum: 329 bytes accounted for. rest might be padding to get a multiple of 
dword as object size (compiler dependent?)

StarObject
size=364 bytes (via sizeof)
is a skyObject => 332 bytes
QString SpType =>   4 bytes
double PM_RA        8 bytes
double PM_Dec       8 bytes
double Parallax     8 bytes
bool Multiplicity   1 byte
bool Variability    1 byte
sum: 362
2 bytes not accounted for, maybe padding to get a multiple of a dword as 
size

This means that about 140 bytes are not accounted for. I am not sure 
whether they can
be completely explained by string lengths of names. The names of most 
stars should be
much simpler.

A big byte consumer is dms, which is used several times. I know little 
to nothing about
our algorithms. To me it makes sense to keep the catalog coordinates in 
double precision.
However I am not so sure whether RA and Dec in skypoint (current true 
sky coordinates)
need to be kept in double precision. I suppose that they are mainly used 
for projection
to a 2d screen and are not used as a base for further calculations. 
Could it be enough
to keep them in float precision by having two dms classes: one in double 
and one in float
precision? This might save 36 bytes per star, which is about 10% of the 
memory load.
And from a

The two dirty flags (2 bits) make up for a total dword in memory 
consumption. This
is unfortunate. But I see no easy way to get around this. One could set 
the sin/cos to
illegal values (like >1), but then checking for dirty would result in a 
floating point operation,
which is too expensive. Perhaps somebody else has an idea? We have 8 dms 
objects in a
StarObject. Each of wich spends 4 bytes on dirty flags, which is 32 
bytes of memory,
or (again) about 10% of the memory load. Do all dms objects get invalid 
at the same time?
Or do some (like the catalog coordinates) never get outdated?
Do we need to keep D and radians in a dms object? Could the algorithms 
be rewritten so that
we only ned one of them (without losing optimizations that we like a lot?)

When I follow where data comes from, then I am tempted to say that the 
dms object is
an expensive way of storing a double value (36 bytes instead of 8). It 
is quite a price to
pay for optimization. Do we need sin/cos from all these values so 
frequently that we
have to precompute them all? Just imagine that 8 bytes were enough: this 
would save
8*28 bytes per star, or 224 bytes, which is half the memory load of kstars.

I think most star objects will only have one name? So we might introduce 
a class of its
own for them? (saves 8 bytes for 2 QStrings, plus the string content, 
just in case
QString does not recognize that one contains the same value as the other?)

The whole area of QString memory management might be worth 
investigating. Maybe it is
not as good as I think?

And all this still leaves some bytes unaccounted for. Are there any 
memory leaks in the
reading of the stars from disk? Files that are not closed etc? Has 
anybody checked for
leaks recently?

By the way: set both faint limits (zoomed in and out) to 1. Close and 
restart kstars.
Go into settings, set limit to 8. The missing data does not get loaded 
in the current cvs...

So much for tonight.

Any comments, ideas etc?

Greetings from Hamburg,

Heiko