Review Request 108914: Thread for OutputModel contents filtering.

Andreas Pakulat apaku at gmx.de
Wed Feb 13 07:24:05 UTC 2013 


> On Feb. 11, 2013, 11:26 p.m., Andreas Pakulat wrote:
> > outputview/outputmodel.cpp, line 59
> > <http://git.reviewboard.kde.org/r/108914/diff/1/?file=113723#file113723line59>
> >
> >     This contradicts the interval-thing, please decide wether you want a single-shot timer that is restarted manually or one that runs in an interval.
> 
> Kevin Funk wrote:
>     Why is that an issue? Interval sets the timeout delay of the timer, single shot defines whether it is restarted or not. They are not mutually exclusive.

Right.


> On Feb. 11, 2013, 11:26 p.m., Andreas Pakulat wrote:
> > outputview/outputmodel.cpp, line 58
> > <http://git.reviewboard.kde.org/r/108914/diff/1/?file=113723#file113723line58>
> >
> >     Why this timer for processing? Whats wrong with blocking the event loop until a set of lines is done?
> 
> Kevin Funk wrote:
>     The idea is that the cache is more likely to be filled with lines, in turn reducing the number of calls to beginInsertRow() later on.

Makes sense, just wondering wether it makes a noticable difference. In particular since most of the time stdout is being read which is usually buffered anyway.


> On Feb. 11, 2013, 11:26 p.m., Andreas Pakulat wrote:
> > outputview/outputmodel.cpp, line 180
> > <http://git.reviewboard.kde.org/r/108914/diff/1/?file=113723#file113723line180>
> >
> >     I can't find anything about this in the Qt docs at the moment, but does QThread actually support "restarting" the thread? I thought I once read that this is not really supported (might work or not), but maybe my recollection is wrong.
> 
> Kevin Funk wrote:
>     Seems to be fine, and worksforme. Related: http://www.qtcentre.org/threads/6548-Multiple-start()-on-same-QThread-Object.

Thats not really an official statement. Would be good to verify that at least the current implementation supports this on *nix (and maybe MacOSX since KDevelop does seem to get used a bit there as well) - or find a unit-test in the qt sources that verifies this works.


> On Feb. 11, 2013, 11:26 p.m., Andreas Pakulat wrote:
> > outputview/outputmodel.cpp, line 199
> > <http://git.reviewboard.kde.org/r/108914/diff/1/?file=113723#file113723line199>
> >
> >     Hmm, what happens if the timer times out, but there are still queued lines in the thread? i.e. the thread is currently working on some lines when its event loop is being exited. Since the event-queue is local in QThread::run() (IIRC) all the still-queued things would be lost in this case.
> >     
> >     So I guess whats needed here is for the outputmodel to keep track of how many lines that where sent to the thread have been finished and if all are done, then start the timer to shut down the thread.
> 
> Kevin Funk wrote:
>     Not an issue. ParseWorker::process() does not return to the event-loop until *all* items were processed.

Thats exactly the problem. While ParseWorker::process() is working a signal can be emitted from appendLines (or rather the invokeMethod call). This will schedule a slot-invocation, but that is executed only once the event loop runs again. Now while process() is working (because there's been lots of lines) the sleeptimer times out and quits the thread. So as soon as process() returns to the event-loop it could exit itself without invoking the appendLines slot. And even if that does happen, if there is only a single line, the worker will schedule another timer which will never fire because the eventloop is going to quit.

So as I said, IMO the sleeptimer should only be started once all lines that have been scheduled for parsing have been received again inside the outputmodel as filtereditems and it should be stopped as soon as new lines come in.


- Andreas


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://git.reviewboard.kde.org/r/108914/#review27247
-----------------------------------------------------------


On Feb. 12, 2013, 2:24 a.m., Kevin Funk wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://git.reviewboard.kde.org/r/108914/
> -----------------------------------------------------------
> 
> (Updated Feb. 12, 2013, 2:24 a.m.)
> 
> 
> Review request for KDevelop.
> 
> 
> Description
> -------
> 
> Quit thread after some time
> 
> Since the OutputModel instance may live for a long time, try not to keep
> a thread running that may never be used again.
> 
> Further improve the background thread
> 
> Cache the lines a certain amount of time before trying to process them.
> This will guarantee that the batch is more likely in a filled state,
> hence reducing the number of calls to beginInsertRow() at the end.
> 
> Also add some debug output for time measurements.
> 
> Fixup/cleanup threaded output line classification.
> 
> - use a QVector instead of a QList for efficiency,
>   reserve/squeeze batch sized buffer
> - use a foreach loop instead of a broken for loop
> - don't use shared pointers for objects that are not
>   shared. here we can simply rely on QObject parent-child
>   relation ship
> - don't use a setupParser init function, use the ctor of the
>   OutputModelPrivate for that
> - use Q_PRIVATE_SLOT to not taint our public interface
> - use invokeMethod to push data to worker instead of signal
>   in public interface
> - properly close and wait for the thread when we destroy the
>   OutputModelPrivate
> - cleanup #include list of outputmodel.h
> 
> Mark FitleredItem as movable and make it possible to sore it in QVector
> 
> 
> Move parsing of lines to a background thread
> 
> This moves the parsing of the added lines to a background thread to avoid
> blocking the UI while the QRegEx'es are being matched. The current
> implementation has a major flaw though, deleting an OutputModel causes
> a crash because the thread cannot be stopped properly (at least thats what
> I think is causing the crash). This might be avoidable by dropping the
> manual usage of QThread and instead do this through QtConcurrent, but then
> we again need manual batching with a invokeMethod to ensure that not all
> lines are added at once to the view.
> 
> Removes 'removeLastLines' since the only user was the ninja-plugin and that
> one should handle its special needs in its own code instead of requiring
> API changes that nobody else needs.
> 
> Removes 'flushLineBuffer' since the idea of that function goes completely
> against the idea of doing batches in the first place so it shouldn't exist.
> If code needs to know when all items are available in the model it'll have
> to connect to the model signals and verify wether the number of lines in
> the model matches the expected final amount. This function is used in the
> ctest-related code in kdevelop, so will need a change there too.
> 
> 
> Diffs
> -----
> 
>   outputview/outputmodel.h 55920beab6ee16643ef449d816d210ba343046e6 
>   outputview/outputmodel.cpp c482222c2448fe75d97911d10aaa7b89fa15bfbf 
>   outputview/outputexecutejob.cpp 56167e6267d5d60caf5378bc1ea994a8a731e789 
>   outputview/filtereditem.h 8eab5e9d6dbefefc203884291f6894ca2a1411a9 
> 
> Diff: http://git.reviewboard.kde.org/r/108914/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Kevin Funk
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kdevelop-devel/attachments/20130213/d490472b/attachment.html>

More information about the KDevelop-devel mailing list