[Kst] 2.0.8

Mon Feb 24 20:48:54 UTC 2014

Hi Barth,

Attached is my log file.

It starts off reading about 2,400 samples per update and slows down to 16,500 samples over an eight 
hour period of logging. Kst is configured to update every 1000 ms.

Do you see this behaviour?

I did not see an instance where Kst read the entire vector, so I think your temporary fix has solved 
that problem.

Ben

On 24/02/2014 8:37 PM, Peter Kümmel wrote:
> On 23.02.2014 21:41, Ben Lewis wrote:
>> Hi Barth,
>>
>> Your fix seems to have improved things some what, but there still seems to be a problem.  It 
>> would be much easier to
>> test if Peter's console log was enabled.
>
> I've added a "Trace" log level which logs the readField() details. You could open Help -> Debug 
> Dialog -> Log
> and leave it open while the data is read.
>
>>
>> Since your fix, I have noticed that there are no long delays between updates. This is great, and 
>> indicates that the
>> problem has been solved. However, the period between updates gets longer as the size of the data 
>> file grows.
>>
>> For example, with an update interval of 1000 ms, I count 40 updates per minute at the start of a 
>> log file and only 6
>> updates per minute after 8 hours of logging. The UI starts off being reasonably responsive 
>> (panning, zooming etc) but
>> after 8 hours it is too slow for practical use. The decline in update rate is noticeable after 
>> only ten minutes of
>> logging and gradually decreases as the file size gets larger.
>>
>> I suspect that if the console log was enabled I would not see the complete read of Vector 1 (as 
>> in the past), but then
>> something else is causing the update periods to slow down and make the system unresponsive.
>>
>> Regards, Ben
>>
>>
>> On 23/02/2014 7:15 AM, Barth Netterfield wrote:
>>> Ben,
>>>
>>> I have pushed a hack.  See if it improves the situation.  If it does, I can
>>> try to clean it up a bit.
>>>
>>> cbn
>>>
>>> On February 22, 2014 1:14:18 PM Barth Netterfield wrote:
>>>
>>>> The fact that the problems happen with the configuration, below, and not on
>>>> my system re-enforces my belief that this is file system race-condition
>>>> related.
>>>>
>>>> kst decides that a file has been re-written/replaced and needs to be re-read
>>>> from the beginning when it has shrunk.  Apparently, in the configuration
>>>> below, if a file is being written to, it can appear smaller than it was.
>>>> As far as I know, this can not happen in Linux.
>>>>
>>>> As a hack (as opposed to a fix, which might require re-writing the file
>>>> system I suspect) we could tell the data vector to delay reading a few
>>>> times if the data source reports back with a shrunk file - and not try a
>>>> re-read until it has failed a few times.  This will (dramatically?) reduce
>>>> the rate of failures but since the problems are unpredictable, it won't
>>>> formally fully fix the problem.
>>>>
>>>> I will commit a hack shortly.
>>>>
>>>> The second problem (reading zeros) is harder to fix (but probably caused by
>>>> the same thing).  I will think about it.
>>>>
>>>> cbn
>>>>
>>>> On February 22, 2014 12:22:47 PM Ben Lewis wrote:
>>>>> I have a USB memory stick plugged into the PC where the data is generated.
>>>>> Data is collected to a RAM buffer and then written to a CSV file on the
>>>>> memory stick. The memory stick has Windows file sharing enabled so that it
>>>>> is accessible over the local network.
>>>>> Kst runs on a different PC. The shared drive (memory stick) on the remote
>>>>> PC is mapped to a local drive. Kst then reads the CSV file as if it were
>>>>> a local file (with update type set to "time interval") The connection
>>>>> between the two PCs is either via a LAN cable or an Ad-hoc WiFi
>>>>> connection. The problem exists under both cases.
>>>>>
>>>>> Remote System (where CSV data file is generated)
>>>>> ----------------------------------------------------------------
>>>>> OS: WindowsXP embedded (32bit)
>>>>> File System: Data is written to a USB memory stick, formatted with NTFS
>>>>>
>>>>> Data Accumulation Rate:
>>>>>        fields/row: 5
>>>>>        characters per field: 7
>>>>>        bytes per row: 41
>>>>>        rows per second: 1500
>>>>>
>>>>> I've attached the first 100 lines of a data file.
>>>>>
>>>>> Local System (where Kst is run)
>>>>> ----------------------------------------
>>>>> OS: Windows7 (64bit)
>>>>> File System: Remote NTFS drive is mapped to local drive
>>>>> Kst: 64bit build
>>>>>
>>>>>>> *   "Out of Memory" error
>>>>>>> http://kde.6490.n7.nabble.com/Out-of-memory-error-td1555215.html The
>>>>>>> error
>>>>>>> message has been improved but the message still appears when it
>>>>>>> shouldn't.
>>>>>> I also can't reproduce this.  Can you give me details (OS, file size)?
>>>>>> Is
>>>>>> the data being updated real time during the read?  At what rate?
>>>>> Same as above
>>>>>
>>>>>>>     * I sometimes get snippets of data missing in a live plot. If I
>>>>>>>     restart
>>>>>>>
>>>>>>> Kst and reload the data there are no missing bits. This seems to happen
>>>>>>> when there is a large amount of network traffic (my data file is not on
>>>>>>> a
>>>>>>> local disk). This is hard to reproduce so it's probably not worth
>>>>>>> worrying
>>>>>>> about at this stage.
>>>>>> Yes.... are you using smb or nfs?  Is it at all correlated with the
>>>>>> first
>>>>>> bug?
>>>>> I'm using Window file sharing.
>>>>> It could very well be related to the first bug but I have no way of
>>>>> telling.
>>
>> _______________________________________________
>> Kst mailing list
>> Kst at kde.org
>> https://mail.kde.org/mailman/listinfo/kst
>>
>
> _______________________________________________
> Kst mailing list
> Kst at kde.org
> https://mail.kde.org/mailman/listinfo/kst

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Kst log.zip
Type: application/zip
Size: 153430 bytes
Desc: not available
URL: <http://mail.kde.org/pipermail/kst/attachments/20140225/3b097b4c/attachment-0001.zip>