Tiles data format

[Dmitry Kazakov]

>
>
> > I think it means that if we make a global compression of the whole data
> > manager with lzf it will be the same 1.6s :)
> And might be about the same compression rate. And is more complex and lose
> the
> possibility to read only certain tiles.
>

ok.


>  > Btw, are you going to disable KoStore compression only for data manager
> or
> > for entire document? I think the latter one is not really fun.
> Only for data manager.
>

ok too =)


>
> > libz allows it for sure. See deflateInit2() in [1]. *
> >
> > *[1] -  http://www.zlib.net/manual.html
>
> Yes, but all ZIP coder/decoder I have seen call deflateInit2 with
> -MAX_WBITS as
> window bits.
>

Though it doesn't mean this is the best choice for compressing tiles =)
Ok, it can be quite difficult to modify KoStore for that.


>  > Hm... wait! Your idea is even better in some way! We need only one
> > KisTileCompressor that uses different abstract compress engines. Yeah.
> You
> > are right with idea of KisAbstractCompressor. But it should provide a
> > couple of additional interfaces:
> >
> > virtual qint32 estimateCompressedSize(qint32 sourceSize); // <- to be
> able
> > to compress tiles in one pass.
> > virtual void adjustForDataSize(qint32 tileDataSize); // <- this one will
> > configure sliding window size (and friends) for particular
> data-stream-size
> >
> > What do you think?
>
> I don't see the point of optimalBufferSize ? or estimateCompressedSize ? In
> our
> case the buffer should be the size of the tile, and the data is saved
> compressed only if the compression result is inferior to the size of the
> tile.
> I don't see the point in making it more complicated.
>

Do these engines make compression in-place? If not we need to create buffer
for this. This buffer should be smaller that the tile-size (now, i'm in
doubt whether there is a guarantee for this from the LZ* for maximum size of
compressed data, should take a look at [2] again). I'm going to have slabs
of compressed tiles, those will be transferred to the disk at once, so i
need to be able to check whether a slab can accept one more tile.

So can we say for sure, that the buffer of some particular size will fit the
compressed tile, before actual compressing?

[2] -
http://www.cs.duke.edu/courses/spring03/cps296.5/papers/ziv_lempel_1977_universal_algorithm.pdf



> Also I don't know how the swapper will work, but for the loading of a file
> it
> is more convenient to have decompressTile(KisTileSP, quint8* buffer); (but
> then
> I would pass data pointer to have an interface that can be easilly used
> outside of the context :) )
>

Well, KisTile takes col, row as arguments in the constructor, and, more than
that, reports about its creation to the memento manager. But the problem is
we don't know new tile's col,row before calling decompressor (as it
encapsulates the header) [3].

So there are two possibilities:
Or we pass all the needed arguments for construction of a tile:
KisTileSP decompressTile(quint8* buffer, KisMementoManager* manager);

or we just pass KisTileHashTable as a factory of tiles that will create
tiles with getTileLazy() - i think this is the best choice:
quint32 decompressTile(quint8* buffer, KisTileHashTable* table);

Btw, we need return value to show how many bytes were read from the stream.



[3] - btw, we can have different KisTileCompressor for every version of kra
;)

-- 
Dmitry Kazakov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.kde.org/pipermail/kimageshop/attachments/20100616/fdc33553/attachment-0001.htm