Words and large files

[Pierre]

On Monday, March 22, 2021 12:24:28 PM CEST Pierre wrote:
> On Monday, March 22, 2021 9:01:21 AM CET Dag wrote:
> > Hi, opened the odf spec in words the other day.
> > This is a document with 800+ pages and a TOC of 60+ pages.
> > I did the same in LO to compare.
> > Don't take the absolute times too seriously as my box is well into its
> > teenage years.
> > But, I think comparison with LO says a lot:
> > 
> > Open: 2,5 mins, LO: 40 secs.
> > Except that TOC page numers is not shown (just ###), navigation, scrolling
> > etc works fine.
> > Save: 4 mins, LO: 30 secs.
> > 
> > And now to the bad part:
> > When a try to type a character words freezes for about 50 secs.
> > It seems the whole doc is re-layouted and also the TOC is updated (page
> > numbers appear).
> > 
> > Then when auto-save kicks in words freezes again without any feedback. I
> > expected to see a status meassage and a progress bar, but no.
> > 
> > Krita solved freeze by making a copy of the doc and save in the
> > background.
> > 
> > LO is better (but not good).
> > Generally you can type new text wo problems but it freezes for shorter
> > periods (maybe when updating page numbers), and it freezes when
> > auto-saving.
> > 
> > It never updates the TOC, this needs to be done manually.
> > 
> > Suggestions (just my 2 cents):
> > 1) TOC should be updated manually to avoid re-layout.
> > 2) Auto-save in the background.
> > 3) Minimize re-layout by e.g. only re-layout dirty pages before and
> > including the displayed page(s).
> 
> Hi
> 
> Thanks for bringing this topic back on the table, long time I did not try to
> do that. On my computer, opening with LO takes 4s, 20s with words… so about
> the same ratio as you have.
> Since my last experiment in this topic, the performance analysis tooling has
> improved so much that this will be a fun thing to do. I'll see what this
> gives.
> And I completely agree with at least parts 1 and 3 of your conclusion. Can
> not tell for the second part, I must think a bit more about it.
> 
> And I disagree with René's conclusions. Having performance issue today
> doesn't mean we can not fix them. And unlike the webbrowser comparison, we
> are not chasing a perpetually moving target with thousands of corporate
> developers adding features on their engines while being 0.1 unpaid
> developer on our engine… It's more like a few unpaid developers against
> less unpaid developers. Still not the best position for us, but far more
> manageable.
> 
>  Pierre

Hi

I've worked a lot on this topic in the past weeks.
On my computer, loading this file is now down to 7s, vs 20s before. I'm sure 
there are other improvements to do.

A big source of slowdown when loading this file is the 2799 bookmarks in the 
file. Our code was (almost) fine, but this is a nightmare regarding 
QTextDocument behaviours. For each bookmark, one QTextCursor is kept is 
KoTextRange, and QTextDocument::insert will go through each cursor to check if 
they need to be moved. This was accounting for about 20% of the load time on 
my computer…
I have opened a bug report on Qt for that, QTBUG-92153, and I will try to 
write a patch. Until this is fixed in Qt (and it'll be some time before 
calligra uses the future Qt 6.x that will have the fix), I've a workaround in 
calligra which was important in beating the 10s milestone.

I've seen other possibly nice performance improvements in lower-level areas. 
For instance, KoXmlNodeData::~KoXmlNodeData is taking more than 100ms… I'm 
sure there is some fun to do there playing with memory allocation heuristics 
and grouping them in a nice way. We could also revisit our compression 
algorithm choice, use more QStringView to reduce memory allocations… If 
anybody is interested… :)

I will try to push readable and reviewable MR in the next week for all the 
changes I've done all over the place.

 Pierre
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.kde.org/pipermail/calligra-devel/attachments/20210404/a9fbd6b4/attachment.sig>