[Kst] branches/work/kst/portto4/kst/src/datasources/ascii

[Peter Kümmel]

On 16.10.2012 23:16, Brisset, Nicolas wrote:
>> SVN commit 1320822 by kuemmel:
>>
>> don't read with mutiple threads from the same file
> Does that mean the speedup you measured applies only to the case where we read from many files simultaneously?
>  From the figures you posted it sounded like you had gotten it to run on a single file...

Here it was also one file, but it must be read as whole file, not with a sliding window.

The ascii plugin reads the file first into memory, then it parses for the data.
Without using a sliding window we have the whole file splitted into several parts
in memory which we could parse in parallel.

But when we use a sliding window, the data is only read from disk when we wanna access
the data while parsing. And when we start multiple threads to parse these parts we
effectively try to read from the same file with several threads, therefore the crash.

When you wanna see the speedup you have to use a file which fits into memory
(don't enable the limitation of the file buffer).

It's possible to also use multiple threads when using a sliding window, but then we
have to split the window and have to start threads on these window parts.
I will add this within the next days.

Peter