Words and large files

Tue Apr 6 10:04:26 BST 2021

On Sunday, 4 April 2021 20:44:27 BST Pierre wrote:
> On Monday, March 22, 2021 12:24:28 PM CEST Pierre wrote:
> > On Monday, March 22, 2021 9:01:21 AM CET Dag wrote:
> > > Hi, opened the odf spec in words the other day.
> > > This is a document with 800+ pages and a TOC of 60+ pages.
> > > I did the same in LO to compare.
> > > Don't take the absolute times too seriously as my box is well into its
> > > teenage years.
> > > But, I think comparison with LO says a lot:
> > > 
> > > Open: 2,5 mins, LO: 40 secs.
> > > Except that TOC page numers is not shown (just ###), navigation,
> > > scrolling
> > > etc works fine.
> > > Save: 4 mins, LO: 30 secs.
> > > 
> > > And now to the bad part:
> > > When a try to type a character words freezes for about 50 secs.
> > > It seems the whole doc is re-layouted and also the TOC is updated (page
> > > numbers appear).
> > > 
> > > Then when auto-save kicks in words freezes again without any feedback. I
> > > expected to see a status meassage and a progress bar, but no.
> > > 
> > > Krita solved freeze by making a copy of the doc and save in the
> > > background.
> > > 
> > > LO is better (but not good).
> > > Generally you can type new text wo problems but it freezes for shorter
> > > periods (maybe when updating page numbers), and it freezes when
> > > auto-saving.
> > > 
> > > It never updates the TOC, this needs to be done manually.
> > > 
> > > Suggestions (just my 2 cents):
> > > 1) TOC should be updated manually to avoid re-layout.
> > > 2) Auto-save in the background.
> > > 3) Minimize re-layout by e.g. only re-layout dirty pages before and
> > > including the displayed page(s).
> > 
> > Hi
> > 
> > Thanks for bringing this topic back on the table, long time I did not try
> > to do that. On my computer, opening with LO takes 4s, 20s with words… so
> > about the same ratio as you have.
> > Since my last experiment in this topic, the performance analysis tooling
> > has improved so much that this will be a fun thing to do. I'll see what
> > this gives.
> > And I completely agree with at least parts 1 and 3 of your conclusion. Can
> > not tell for the second part, I must think a bit more about it.
> > 
> > And I disagree with René's conclusions. Having performance issue today
> > doesn't mean we can not fix them. And unlike the webbrowser comparison, we
> > are not chasing a perpetually moving target with thousands of corporate
> > developers adding features on their engines while being 0.1 unpaid
> > developer on our engine… It's more like a few unpaid developers against
> > less unpaid developers. Still not the best position for us, but far more
> > manageable.
> > 
> >  Pierre
> 
> Hi
> 
> I've worked a lot on this topic in the past weeks.
> On my computer, loading this file is now down to 7s, vs 20s before. I'm sure
> there are other improvements to do.
> 
> A big source of slowdown when loading this file is the 2799 bookmarks in the
> file. Our code was (almost) fine, but this is a nightmare regarding
> QTextDocument behaviours. For each bookmark, one QTextCursor is kept is
> KoTextRange, and QTextDocument::insert will go through each cursor to check
> if they need to be moved. This was accounting for about 20% of the load
> time on my computer…
> I have opened a bug report on Qt for that, QTBUG-92153, and I will try to
> write a patch. Until this is fixed in Qt (and it'll be some time before
> calligra uses the future Qt 6.x that will have the fix), I've a workaround
> in calligra which was important in beating the 10s milestone.
> 
> I've seen other possibly nice performance improvements in lower-level areas.
> For instance, KoXmlNodeData::~KoXmlNodeData is taking more than 100ms… I'm
> sure there is some fun to do there playing with memory allocation
> heuristics and grouping them in a nice way. We could also revisit our
> compression algorithm choice, use more QStringView to reduce memory
> allocations… If anybody is interested… :)
> 
> I will try to push readable and reviewable MR in the next week for all the
> changes I've done all over the place.

  Some absolutely stellar work going on here! Thank you so much for taking on 
this beast :)

>  Pierre

-- 
..dan / leinir..
http://leinir.dk/