making kioslave/trash size cache more robust

[Diggory Hardy]

On 5 December 2012 22:35, David Faure <faure at kde.org> wrote:

> On Friday 23 November 2012 02:58:36 Diggory Hardy wrote:
> > Dear list,
> >
> > Since this bug — https://bugs.kde.org/show_bug.cgi?id=245482 — has not
> been
> > addressed and since it makes KDE's trash somewhat incompatible with other
> > implementations, I thought I'd have a go at addressing it.
>
> Great.
>
> > First, does anyone besides KDE use the ~/.local/share/Trash/metadata
> file?
> > It's not mentioned in the specification[1].
>
> Right, it's not.
>
> I actually can't remember (nor find on google) whether an extension to the
> trash spec was proposed on the freedesktop xdg list for this. Tobias, do
> you
> remember?
>
> > Second, what's the best way to solve the problem? It probably mostly
> occurs
> > because other trash implementations don't adjust the metadata file.
>
> Yes, so this should really be the first approach to this problem: ensuring
> that
> all trash implementations collaborate in storing the size of the trash [in
> each trash dir].
>

So you're suggesting that we (or someone) should somehow bring all
implementations in line with an updated spec? How would one go about doing
this? My aim was to make what KDE does more robust in the face of differing
implementations (as well as problems caused by non-atomic writes), although
updating the spec at the same time sounds like a good idea.

>
> > I can think of a couple of workarounds for this:
> >
> > 1) Store the size of each item in a line in the 'metadata' file.
> > (Can we use the filename as a KConfig key?)
>
> Sounds fine, although you might uncover a few missing escapes in the
> KConfig
> code when doing that, in corner cases (filenames containing [, ] or =).
>

Right. If anyone knows more about this please let me know.

>
> > To get size, first synchronise lines against the 'files' dir (without
> > recalculating when names match), then sum the sizes.
> > Mostly this should be pretty fast I think.
>
> It could be, provided that parsing is faster than stat()ing. Maybe true
> with
> cold caches, but not sure it's still true the second time... This would
> have
> to be measured, to avoid choosing a solution that actually leads to worse
> performance.
>

I have no idea how file systems store these stats, though I imagine some
implementations might require quite a few seeks to stat all the files.
Since CPUs are pretty fast I don't think parsing would ever be worse than
bad cases of stat-ing (e.g. several thousand items on a slow HDD), which is
I presume the ones we're trying to optimise.

>
> > 2) Store the size of each item in the corresponding 'info' file. Should
> be
> > even more robust (we just need to add the line if it wasn't there
> before),
> > might be a good candidate for extending the current specification[1], and
> > not very likely to cause problems — except that there's a lot of small
> > files to read to get the total size, so it may not be that fast in all
> > cases (1000 items on an old HD? NFS/SMB?).
>
> This sounds slower than simply stat()ing every trashed filed to determine
> its
> size. stat() is a single syscall, while open+read+parsing+close is a lot
> more...
> This should IMHO neither be done as a workaround nor as part of the spec.
>

Probably true that this isn't a good solution, although don't forget that a
trashed item could be a directory containing many files so it should still
be faster than bad cases (the problem with benchmarking is that you don't
know exactly what someone will have in trash).

>
> --
> David Faure, faure at kde.org, http://www.davidfaure.fr
> Working on KDE, in particular KDE Frameworks 5
>
>
So, I'll wait for a bit more input before really starting. I'm a bit busy
at the moment anyway.

Regards,

Diggory
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-core-devel/attachments/20121206/9a919310/attachment.htm>