[Kst] 2.0.8

Mon Feb 24 11:33:59 UTC 2014

Hi Barth,

Please see my comments below.

Regards, Ben

On 24/02/2014 7:53 AM, barth.netterfield at gmail.com wrote:
> If you exit and restart kst does that revert to good behavior?
I don't think restarting Kst helps. In the chart below I restarted Kst between the second last and 
last data point. The overall trend is still slowing.

>
> Are you in count from end mode (how many samples?) or read to end (how much of the file?)?
I start at the first sample and read to the end. i.e. I read 100% of the file. I have 24 GB of RAM 
which should allow me to do this.
>
> I'll try to reproduce the slowdown here.
>
>
> Barth Netterfield
> 416-845-0946
>    Original Message
> From: Ben Lewis
> Sent: Sunday, February 23, 2014 3:41 PM
> To: netterfield at astro.utoronto.ca
> Reply To: kst at kde.org
> Cc: kst at kde.org
> Subject: Re: [Kst] 2.0.8
>
> Hi Barth,
>
> Your fix seems to have improved things some what, but there still seems to be a problem. It would
> be much easier to test if Peter's console log was enabled.
>
> Since your fix, I have noticed that there are no long delays between updates. This is great, and
> indicates that the problem has been solved. However, the period between updates gets longer as the
> size of the data file grows.
>
> For example, with an update interval of 1000 ms, I count 40 updates per minute at the start of a log
> file and only 6 updates per minute after 8 hours of logging. The UI starts off being reasonably
> responsive (panning, zooming etc) but after 8 hours it is too slow for practical use. The decline in
> update rate is noticeable after only ten minutes of logging and gradually decreases as the file size
> gets larger.
>
> I suspect that if the console log was enabled I would not see the complete read of Vector 1 (as in
> the past), but then something else is causing the update periods to slow down and make the system
> unresponsive.
>
> Regards, Ben
>
>
> On 23/02/2014 7:15 AM, Barth Netterfield wrote:
>> Ben,
>>
>> I have pushed a hack. See if it improves the situation. If it does, I can
>> try to clean it up a bit.
>>
>> cbn
>>
>> On February 22, 2014 1:14:18 PM Barth Netterfield wrote:
>>
>>> The fact that the problems happen with the configuration, below, and not on
>>> my system re-enforces my belief that this is file system race-condition
>>> related.
>>>
>>> kst decides that a file has been re-written/replaced and needs to be re-read
>>> from the beginning when it has shrunk. Apparently, in the configuration
>>> below, if a file is being written to, it can appear smaller than it was.
>>> As far as I know, this can not happen in Linux.
>>>
>>> As a hack (as opposed to a fix, which might require re-writing the file
>>> system I suspect) we could tell the data vector to delay reading a few
>>> times if the data source reports back with a shrunk file - and not try a
>>> re-read until it has failed a few times. This will (dramatically?) reduce
>>> the rate of failures but since the problems are unpredictable, it won't
>>> formally fully fix the problem.
>>>
>>> I will commit a hack shortly.
>>>
>>> The second problem (reading zeros) is harder to fix (but probably caused by
>>> the same thing). I will think about it.
>>>
>>> cbn
>>>
>>> On February 22, 2014 12:22:47 PM Ben Lewis wrote:
>>>> I have a USB memory stick plugged into the PC where the data is generated.
>>>> Data is collected to a RAM buffer and then written to a CSV file on the
>>>> memory stick. The memory stick has Windows file sharing enabled so that it
>>>> is accessible over the local network.
>>>> Kst runs on a different PC. The shared drive (memory stick) on the remote
>>>> PC is mapped to a local drive. Kst then reads the CSV file as if it were
>>>> a local file (with update type set to "time interval") The connection
>>>> between the two PCs is either via a LAN cable or an Ad-hoc WiFi
>>>> connection. The problem exists under both cases.
>>>>
>>>> Remote System (where CSV data file is generated)
>>>> ----------------------------------------------------------------
>>>> OS: WindowsXP embedded (32bit)
>>>> File System: Data is written to a USB memory stick, formatted with NTFS
>>>>
>>>> Data Accumulation Rate:
>>>> fields/row: 5
>>>> characters per field: 7
>>>> bytes per row: 41
>>>> rows per second: 1500
>>>>
>>>> I've attached the first 100 lines of a data file.
>>>>
>>>> Local System (where Kst is run)
>>>> ----------------------------------------
>>>> OS: Windows7 (64bit)
>>>> File System: Remote NTFS drive is mapped to local drive
>>>> Kst: 64bit build
>>>>
>>>>>> * "Out of Memory" error
>>>>>> http://kde.6490.n7.nabble.com/Out-of-memory-error-td1555215.html The
>>>>>> error
>>>>>> message has been improved but the message still appears when it
>>>>>> shouldn't.
>>>>> I also can't reproduce this. Can you give me details (OS, file size)?
>>>>> Is
>>>>> the data being updated real time during the read? At what rate?
>>>> Same as above
>>>>
>>>>>> * I sometimes get snippets of data missing in a live plot. If I
>>>>>> restart
>>>>>>
>>>>>> Kst and reload the data there are no missing bits. This seems to happen
>>>>>> when there is a large amount of network traffic (my data file is not on
>>>>>> a
>>>>>> local disk). This is hard to reproduce so it's probably not worth
>>>>>> worrying
>>>>>> about at this stage.
>>>>> Yes.... are you using smb or nfs? Is it at all correlated with the
>>>>> first
>>>>> bug?
>>>> I'm using Window file sharing.
>>>> It could very well be related to the first bug but I have no way of
>>>> telling.
> _______________________________________________
> Kst mailing list
> Kst at kde.org
> https://mail.kde.org/mailman/listinfo/kst
> _______________________________________________
> Kst mailing list
> Kst at kde.org
> https://mail.kde.org/mailman/listinfo/kst

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kst/attachments/20140224/bc1bcc66/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cafdjdba.png
Type: image/png
Size: 7000 bytes
Desc: not available
URL: <http://mail.kde.org/pipermail/kst/attachments/20140224/bc1bcc66/attachment.png>