[Kst] 2.0.8

Mon Feb 24 12:02:07 UTC 2014

Thanks Peter,

It works great. I'll do some more testing and report back if I find anything.

Ben

On 24/02/2014 8:37 PM, Peter Kümmel wrote:
> On 23.02.2014 21:41, Ben Lewis wrote:
>> Hi Barth,
>>
>> Your fix seems to have improved things some what, but there still seems to be a problem.  It 
>> would be much easier to
>> test if Peter's console log was enabled.
>
> I've added a "Trace" log level which logs the readField() details. You could open Help -> Debug 
> Dialog -> Log
> and leave it open while the data is read.
>
>>
>> Since your fix, I have noticed that there are no long delays between updates. This is great, and 
>> indicates that the
>> problem has been solved. However, the period between updates gets longer as the size of the data 
>> file grows.
>>
>> For example, with an update interval of 1000 ms, I count 40 updates per minute at the start of a 
>> log file and only 6
>> updates per minute after 8 hours of logging. The UI starts off being reasonably responsive 
>> (panning, zooming etc) but
>> after 8 hours it is too slow for practical use. The decline in update rate is noticeable after 
>> only ten minutes of
>> logging and gradually decreases as the file size gets larger.
>>
>> I suspect that if the console log was enabled I would not see the complete read of Vector 1 (as 
>> in the past), but then
>> something else is causing the update periods to slow down and make the system unresponsive.
>>
>> Regards, Ben
>>
>>
>> On 23/02/2014 7:15 AM, Barth Netterfield wrote:
>>> Ben,
>>>
>>> I have pushed a hack.  See if it improves the situation.  If it does, I can
>>> try to clean it up a bit.
>>>
>>> cbn
>>>
>>> On February 22, 2014 1:14:18 PM Barth Netterfield wrote:
>>>
>>>> The fact that the problems happen with the configuration, below, and not on
>>>> my system re-enforces my belief that this is file system race-condition
>>>> related.
>>>>
>>>> kst decides that a file has been re-written/replaced and needs to be re-read
>>>> from the beginning when it has shrunk.  Apparently, in the configuration
>>>> below, if a file is being written to, it can appear smaller than it was.
>>>> As far as I know, this can not happen in Linux.
>>>>
>>>> As a hack (as opposed to a fix, which might require re-writing the file
>>>> system I suspect) we could tell the data vector to delay reading a few
>>>> times if the data source reports back with a shrunk file - and not try a
>>>> re-read until it has failed a few times.  This will (dramatically?) reduce
>>>> the rate of failures but since the problems are unpredictable, it won't
>>>> formally fully fix the problem.
>>>>
>>>> I will commit a hack shortly.
>>>>
>>>> The second problem (reading zeros) is harder to fix (but probably caused by
>>>> the same thing).  I will think about it.
>>>>
>>>> cbn
>>>>
>>>> On February 22, 2014 12:22:47 PM Ben Lewis wrote:
>>>>> I have a USB memory stick plugged into the PC where the data is generated.
>>>>> Data is collected to a RAM buffer and then written to a CSV file on the
>>>>> memory stick. The memory stick has Windows file sharing enabled so that it
>>>>> is accessible over the local network.
>>>>> Kst runs on a different PC. The shared drive (memory stick) on the remote
>>>>> PC is mapped to a local drive. Kst then reads the CSV file as if it were
>>>>> a local file (with update type set to "time interval") The connection
>>>>> between the two PCs is either via a LAN cable or an Ad-hoc WiFi
>>>>> connection. The problem exists under both cases.
>>>>>
>>>>> Remote System (where CSV data file is generated)
>>>>> ----------------------------------------------------------------
>>>>> OS: WindowsXP embedded (32bit)
>>>>> File System: Data is written to a USB memory stick, formatted with NTFS
>>>>>
>>>>> Data Accumulation Rate:
>>>>>        fields/row: 5
>>>>>        characters per field: 7
>>>>>        bytes per row: 41
>>>>>        rows per second: 1500
>>>>>
>>>>> I've attached the first 100 lines of a data file.
>>>>>
>>>>> Local System (where Kst is run)
>>>>> ----------------------------------------
>>>>> OS: Windows7 (64bit)
>>>>> File System: Remote NTFS drive is mapped to local drive
>>>>> Kst: 64bit build
>>>>>
>>>>>>> *   "Out of Memory" error
>>>>>>> http://kde.6490.n7.nabble.com/Out-of-memory-error-td1555215.html The
>>>>>>> error
>>>>>>> message has been improved but the message still appears when it
>>>>>>> shouldn't.
>>>>>> I also can't reproduce this.  Can you give me details (OS, file size)?
>>>>>> Is
>>>>>> the data being updated real time during the read?  At what rate?
>>>>> Same as above
>>>>>
>>>>>>>     * I sometimes get snippets of data missing in a live plot. If I
>>>>>>>     restart
>>>>>>>
>>>>>>> Kst and reload the data there are no missing bits. This seems to happen
>>>>>>> when there is a large amount of network traffic (my data file is not on
>>>>>>> a
>>>>>>> local disk). This is hard to reproduce so it's probably not worth
>>>>>>> worrying
>>>>>>> about at this stage.
>>>>>> Yes.... are you using smb or nfs?  Is it at all correlated with the
>>>>>> first
>>>>>> bug?
>>>>> I'm using Window file sharing.
>>>>> It could very well be related to the first bug but I have no way of
>>>>> telling.
>>
>> _______________________________________________
>> Kst mailing list
>> Kst at kde.org
>> https://mail.kde.org/mailman/listinfo/kst
>>
>
> _______________________________________________
> Kst mailing list
> Kst at kde.org
> https://mail.kde.org/mailman/listinfo/kst