Dependencies of parse-jobs

[Sven Brauch]

Hey there,

no, I absolutely can't get it to work with priorities and the document
queue. I don't think this issue can be solved by just setting
priorities and trying to schedule documents in the right way... with
multiple threads the ordering is never really guaranteed.
I also can't get it to immediately start a parseJob at a specified
point (what function do I need to call?)...

I'm wondering what the good way of dealing with this would be. What
about attaching a list of required top-contexts to each file and
change the parse-job creation logic to take that into account?
Shouldn't be much work to implement that.

Greetings,
Sven

Am 13. Februar 2012 16:06 schrieb Sven Brauch <svenbrauch at googlemail.com>:
> Hi,
>
> yes, I guess it's something like that too (race-condition or similar).
> However I thought I could reproduce this with only one parser-thread
> too... not sure.
> I'll try forcing the parse-job to run immediately after an include is
> encountered, thus delaying its parent; I wanted to avoid this, as I
> think it'll really increase memory consumption. But if that's the only
> way to make it work correctly, that'll have to do :)
>
> I'll report back to you as soon as I can see whether this works or not.
>
> Greetings,
> Sven
>
> Am 13. Februar 2012 10:54 schrieb David Nolden <zwabel at googlemail.com>:
>> 2012/2/12 Sven Brauch <svenbrauch at googlemail.com>:
>>> Hi there,
>>>
>>> PovAddict and I were tracking a bug (in the python plugin) a few days
>>> ago which turned out to be caused by the order in which documents are
>>> parsed to be wrong (as in, not like I wanted it to be, heh).
>>> Currently, I'm doing this: If document A is parsed and an "import"
>>> (same like #include in c++) is encountered, a parse job is created for
>>> the imported document using
>>>    DUChain::self()->updateContextForUrl(...)
>>> with a priority that has a lower value than the current parse job (and
>>> thus a higher priority (who invented this?)). Then the current
>>
>> This one was my fault, I followed the linux priority logic, which
>> later showed to suck hard. :-)
>>
>>> document is marked as "needs update", and after the parse job
>>> finished, it's registered for a re-parse again (that's done inside
>>> parsejob::run) with a lower priority than it had before (I'll use the
>>> word "priority" as everybody expects it to be used from now on):
>>>    KDevelop::ICore::self()->languageController()->backgroundParser()->addDocument(...)
>>> Now, that roughly works, but not quite; sometimes, the documents are
>>> not parsed exactly in the order their priorities suggests. I guess
>>> that is to be expected -- but what else is there except priorities
>>> that can be used to "sort" parse jobs?
>>>
>>> Here's the TL;DR version: What's the "good" way of forcing document A
>>> to be re-parsed as soon as there's a top-context for B available which
>>> satisfies the required $minimum_features? How does C++ do this? I
>>> looked into the code, but couldn't quite find anything related to that
>>> (it's a huge project...).
>>>
>>> Any help would be appreciated. :)
>>>
>>> Greetings,
>>> Sven
>>
>> In C++, included documents may depend on other documents included
>> earlier, and thus they must be parsed right in-place.
>>
>> You could do the same, by simply creating a new parse-job in-place and
>> starting it recursively. You just need to care about preventing
>> infinite recursion and multiple running parse-jobs für the same
>> document. The question is what would be the correct way to deal with
>> such recursion.
>>
>> In a language where such recursion is allowed, the correct solution
>> would probably be a multi-pass parsing: When you encounter an include,
>> parse it in a "simplified" mode (just creating declarations for the
>> symbol-table, but without building uses and/or types etc.), and later
>> re-parse it in full mode after the "simplified" mode has been finished
>> for all includes.
>>
>> Greetings, David
>>
>> --
>> KDevelop-devel mailing list
>> KDevelop-devel at kdevelop.org
>> https://barney.cs.uni-potsdam.de/mailman/listinfo/kdevelop-devel