Tiles data format

[Dmitry Kazakov]

>
> * .kra with compressed tiles (lzf) and no global compression: 67MB, saving
> time 1.6s
>


* .kra with uncompressed tiles and global compression: 28MB, saving time 28s
> * .kra with compressed tiles using deflate and no global compression: 27MB
> saving time 23s
>
^^ both 'deflate'?

I think it means that if we make a global compression of the whole data
manager with lzf it will be the same 1.6s :)

Btw, are you going to disable KoStore compression only for data manager or
for entire document? I think the latter one is not really fun.


>
> > Well, first, there is an abstract compressor should be done first, to
> will
> > allow swapper to save data in the same format as kra. And life of both of
> > us will be much easier =)
> I don't think there is an obligation of using the same algorithm for both
> solution. The requirement for saving is the fastest is the best.
>

Absolutely the same requirement for swapping =)


>  > And second, i'm not sure we interpreted the results of those tests
> right.
> > Sliding window size of zlib is configurable and it is big by default, so
> it
> > could cause the difference.
> >
> > If we interpreted them wrong, then it's better to replace (configure)
> > compression of the KoStore, than to do this change.
> Except there is no API to do that (and it uses KDE stuff, so no easy change
> here). And assuming the ZIP file format even allows that.
>

libz allows it for sure. See deflateInit2() in [1]. *

*[1] -  http://www.zlib.net/manual.html



> > PS:
> > Anyway, if you wish, i can hurry with this abstract compressor a bit. =)
>
> class AbstrstractCompress { virtual compress(void* data, int data_length,
> void* output, int output_length) = 0; virtual decompress(void* data, int
> data_length, void* output, int output_length) = 0;   }; ? Or am I missing
> something ?
>
> or compress(void* data, int data_length, void* output, int output_length,
> CompressionMethod)
>
> Well anyway, the new format for tiles allows room for experimentation, so I
> will implement that, and then we can play with it.
>

Something like that, but a bit more high-level. (do we need fast compression
for the stuff other than tiles?) I want to encapsulate tile header into it.
(the one that puts origin of the tile, it's size and etc). Something like:

class KisTileCompressor {
    qint32 optimalBufferSize();     // for LZ77 - there is a formula for
estimating high boundary for compressed size
    void compressTile(KisTileSP tile, quint8* buffer);
    KisTileSP decompressTile(quint8* buffer);
};

In such a case i will use the same binary file format for swapper, this will
allow simply copy swapper's data to the kra file when saving.

Hm... wait! Your idea is even better in some way! We need only one
KisTileCompressor that uses different abstract compress engines. Yeah. You
are right with idea of KisAbstractCompressor. But it should provide a couple
of additional interfaces:

virtual qint32 estimateCompressedSize(qint32 sourceSize); // <- to be able
to compress tiles in one pass.
virtual void adjustForDataSize(qint32 tileDataSize); // <- this one will
configure sliding window size (and friends) for particular data-stream-size

What do you think?

-- 
Dmitry Kazakov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.kde.org/pipermail/kimageshop/attachments/20100616/2516f6dc/attachment.htm