UDSEntry compression ideas

[Mark]

Hi All,

please read this as a brainstorm post with some ideas. I have no code
working in this area yet.

Lazy loading is awesome, but if you can compress the data enough that you
still have all details without lazy loading is even better i think. For
example, doing a directory listing on any given folder gives at least the
following UDSEntry values:
- Device ID
- Inode
- User
- Group

In the "experimental" case where you list the folder contents that has just
a bunch of files created by one user you can say that at least the above 4
properties are the same for all files. However, the experimental situation
certainly doesn't work on "real data" where you can have files from
multiple users and folders as well. Folders will likely have a different
inode number (not tested). However, this is an area where - in most cases -
half of the data (by default we only have 8 UDSEntry properties) is
redundant. Among the 4 others are also two timestamps which are very often
the same as well.

I'd like to play with compression here, but i'm not sure how to follow up
on that. One idea i'm having is a multikey - > single value storage
structure which is not in C++ or Qt by itself. Perhaps boost::multiindex
would be a possible solution here? Where the index would probably be the
filename which you can then also remove from the UDSEntry since you can get
from the keys themselves.

Other then that i don't know a sane way of doing this or even if this will
work at all. The above idea would already require a singlethon
"UDSEntryDataContainer" that all UDSEntry objects would use. The UDSEntry
API would remain the same, the implementation would differ quite a lot.

I also don't know how much data will be saved at all by doing this. Sure,
it will save "some" but i image you will get some more pointers with a
multiindex structure and perhaps some other container bookkeeping code?

What's your idea here? If you have a better/other idea, please do tell :)

I'm looking forward to your input and other far better ideas!

Kind regards,
Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.kde.org/mailman/private/kfm-devel/attachments/20130916/9e81af9f/attachment.htm>