UDSEntry compression ideas

[Mark]

On Tue, Sep 17, 2013 at 1:58 PM, Todd <toddrjen at gmail.com> wrote:

>
> On Sep 17, 2013 12:58 AM, "Frank Reininghaus" <frank78ac at googlemail.com>
> wrote:
> >
> > Hi Mark,
> >
> > I'm too tired for a longer reply already, but here is a quick idea
> > that I just had and tried immediately.
> >
> > 2013/9/16 Mark:
> > > Hi All,
> > >
> > > please read this as a brainstorm post with some ideas. I have no code
> > > working in this area yet.
> > >
> > > Lazy loading is awesome, but if you can compress the data enough that
> you
> > > still have all details without lazy loading is even better i think. For
> > > example, doing a directory listing on any given folder gives at least
> the
> > > following UDSEntry values:
> > > - Device ID
> > > - Inode
> > > - User
> > > - Group
> > >
> > > In the "experimental" case where you list the folder contents that has
> just
> > > a bunch of files created by one user you can say that at least the
> above 4
> > > properties are the same for all files. However, the experimental
> situation
> > > certainly doesn't work on "real data" where you can have files from
> multiple
> > > users and folders as well.
> >
> > Well, I guess that most people have quite a few folders where the
> > "User" and "Group" are the same for all files ;-)
> >
> > The easiest "compression" that I could think of was to force the
> > UDSEntries to implicitly share all their QStrings for one field if
> > they are all equal:
> >
> > http://paste.kde.org/p52a24b49/
> >
> > Reduces the memory consumption in Details View for 100,000 items from
> > 160.5 MB to 147 MB here. The patch is a bit hackish though, and it
> > might have a small performance penalty. Maybe it's worth investigating
> > that further though.
> >
> > I guess any more sophisticated "compression" approach will have to be
> > implemented in frameworks only.
> >
> > Cheers,
> > Frank
>
> Do we need the UID and GID?  Can we just store whether the file is
> readable, writable, and executable? That would be three bools rather than
> two ints.
>
I don't know, but that would be a saving on both the client side as well as
the slave side..
Perhaps we need to invite David to chime in and give his view on this?
Added him to the CC.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.kde.org/mailman/private/kfm-devel/attachments/20130917/23bd5abe/attachment.htm>