Find uses/rename is being painfully slow

[Dmitry Risenberg]

2011/3/1 David Nolden <david.nolden.kdevelop at art-master.de>:
> 2011/3/1 Dmitry Risenberg <dmitry.risenberg at gmail.com>:
>>>> I've tried adding the document for parsing twice - simple parsing with
>>>> normal priority and full parsing with low priority. This way renaming
>>>> speeds up a lot.
>>> I cannot see how this could speed up the renaming. Before uses can be
>>> searched, the _full_ parsed result is required anyway.
>>
>> The full parsing is performed in background when there are no other
>> parsing jobs, so that the user won't have to wait for it when he calls
>> for a rename.
>
> But the user has to wait until the rename is finished anyway.. also he
> wants to see a preview of the items are will be renamed, so where's
> the gain?

Currently all files are reparsed with AllDeclarationsContextsAndUses
when the renaming is triggered. But this parsing can be done on
startup, after simple parsing has finished, so when the renaming is
triggered, only the changed files have to be reparsed while the rest
of files will be up to date - this is much faster than reparsing
everything.

>
>>>> However, there is still a problem - some of the files are reparsed too
>>>> often (on every startup, even if they are not changed), and this kills
>>>> most of the gained benefits. I'm trying to debug this issue, but it is
>>>> complicated, because there are no unit tests for background parsing
>>>> and duchain. Looks like the call for 'featuresSatisfied' on
>>>> preprocessjob.cpp:199 is failing, but I can't track why.
>>>>
>>>> So the question is - how can I debug duchain-related code and see what
>>>> is stored in duchain - which file, with what features, when it was
>>>> parsed, etc.? And is there any special way to debug updating-related
>>>> issues?
>>>
>>> We'll have to live with the fact that we need to re-parse nearly
>>> always. For example, it's enough if you open one central header-file,
>>> insert one character and remove it again (the revision is now
>>> changed), now you start your rename, parse everything, and when you
>>> re-start kdevelop you will have to re-parse everything again, because
>>> the document revision is different.
>>
>> I am aware of that, but I mean opening/closing without changing
>> anything - some open files still get reparsed, and mostly the same
>> each time. I want to fix that, because it may become costly if full
>> parsing is performed.
>
> I don't exactly remember, but maybe we assign different revisions to
> open documents than to closed documents (depends on kate). We probably
> should always force the revision to zero for documents without a
> difference to disk.

Looks like I found the cause of problem - it is preprocessjob.cpp:199:

if(updatingEnvironmentFile->featuresSatisfied(parentJob()->minimumFeatures())
&& updatingEnvironmentFile->featuresSatisfied(parentJob()->slaveMinimumFeatures()))

The bad case is when slaveMinimumFeatures() are greater than
minimumFeatures() - minimumFeatures() eventually get stored in the
duchain and the next time the second condition fails again, causing a
reparse. I see two ways of fixing that: either remove the second check
(seems inappropriate here anyway, because it is checked in
sourceNeeded()) or do minimumFeatures |= slaveMinimumFeatures,
probably except the ForceUpdate set of flags.

>
>> By the way, also tried QtCreator today - it is much faster in renaming.
>
> We can reach the biggest gain by doing a pre-grep to filter out the
> candidate files beforehand, as I wrote before. I will try this now.
>
>>> In future, we should consider using something like "semantic
>>> revisions", eg. special persistent revisions which we only increase
>>> for a document when a part of its public interface has changed (a
>>> macro, public function or public class was changed). Right now, we
>>> basically do it the same was as gcc, which only looks at
>>> modification-times (from what I know).
>>>
>>> The updating stuff can be debugged, by commenting out
>>> DEBUG_NEEDSUPDATE in modificationrevisionset.cpp. However, I debugged
>>> this code many times, and most probably the only problem are these
>>> document revisions.
>>
>> I'll try it, but I also want to check "dumping to disk", particularly
>> the m_features members in ParsingEnvironmentData and TopDUContextData.
>> Could not find the saving code anywhere.
> It has no saving code, it is mapped directly onto disk.
>

-- 
Dmitry Risenberg