[Kst] Trending from daily CSV Files KST-Plot Windows Version

Wed Oct 14 15:50:27 UTC 2015

Hello Barth, Hello KstPlot Team,

Thanks for your answer.

I changed configuration to 'count from end' which now works great and fast!

In future I will never remove files from neither 'list of files' nor delete CSV files from hard disk.

This will work. See further comments below *

A nice feature for a new release would be to select optionally range of data via date/time picker.

This will give user possibility to select data in a defined range (not always read to end)

I guess date/time entry is already possible as tooltip suggests this feature.

I tried different formats but I didn’t get it running. Maybe formats I’ve entered were wrong.

Definitely it would be great to select the x-axis range also via date/time picker, which would be more

user-friendly and prevent wrong inputs.

[cid:image003.png at 01D106A6.27079080]

[cid:image002.png at 01D106A8.CF4C6DE0]

===================================================================================

*

At the moment CSV files will be zipped after a certain time (e.g. 90days) and send to remote storage server.

At the same time those CSV files will be deleted.

This is not a problem as long as Kst2Plot will not be restarted. After restart Kst2Plot

files which are no longer on hard disk will be reported - see dialog below (in German, sorry)

[cid:image006.png at 01D106A8.CF4C6DE0]

File not found => skip

After skip => datasource files reported from Kst2Plot looks like screenshot below

Last row entry = “list of files’ itself, which is strange

After that Kst2plot data-source must be reconfigured  via data wizard.

[cid:image004.png at 01D106A4.35B7DBA0]

Thanks a lot

Best regards

Jörg

-----Ursprüngliche Nachricht-----
Von: Barth Netterfield [mailto:barth.netterfield at utoronto.ca]
Gesendet: Mittwoch, 30. September 2015 16:06
An: Huber, Jörg
Cc: kst at kde.org
Betreff: Re: AW: [Kst] Trending from daily CSV Files KST-Plot Windows Version

Sorry for the slow reply.  I'm finally back.

The intended workflow (and the mode we operate in) is to never remove files from the beginning of the 'list of files' and to instead operate in 'count from end'

mode instead of 0 to end mode.  The number of frames listed in 'count from end' should be whatever is needed for the past 30/60/90 days.  This should be fast.

The data will scroll by as new data comes.

Does this work for you?

cbn

On September 22, 2015 10:33:59 AM Huber, Jörg wrote:

> Hello Barth,

>

> Thanks a lot for your answers and help!

> I was on holiday, so sorry for the delay in my reply.

>

> I did some more tests and I must correct my statement:

> If data vector(s) is/are configured "read from start = 0 to end"

>                 => 1. plots are updated if one or more files are

> deleted in KstPlot “list of files” e.g. 1st entry = oldest file of 30

> files is deleted at the beginning of "list of files"

>                => 2. the same happens if one file is add to "list of files"

>

> “list of files”

>

>

>

>

> The difference between case 1 + 2 is as described below:

>

> case 2: plot update is fast as additional data are added to existing

> plot

> (<10sec) which is perfect!

> case 1: plot update is slow as additional data are delete from

> existing plot Depending on how many files in “list of files” and data points (here:

> 30files à 80.000 data points) Window of KstPlot is frozen as data

> curve / plot must be updated or re-arranged from the beginning. This

> takes in my case about 10/20/30 min with 30/60/90 files.

>                 At this time user interface of KstPlot and online charts

> will be frozen and no longer actualized.

> Is there a possibility to solve this problem? Maybe using additional

> “task/background worker” for data vector update? I’m not familiar with

> phyton scripting but please let me know, if and how the problem could

> be managed by phyton script in a later release of KstPlot.

>

> Regarding:  file dialog messages: it seems that "clean up non existing

> files" fixed it!

> Thanks a lot

>

> Best regards

> Jörg

>

>

>

> Best regards

>

> Jörg Huber

> Graduate Engineer

> electrical engineering

> ____________________________________________________

>

> Herrenknecht AG

> Schlehenweg 2

> 77963 Schwanau-Allmannsweier

> Germany

> (+49 (7824) 302 5440

> 7+49 (7824) 302 2940

>

> mailto:huber.joerg at herrenknecht.de

> http://www.herrenknecht.de

>

>

>

>

>

> -----Ursprüngliche Nachricht-----

> Von: Barth Netterfield [mailto:barth.netterfield at utoronto.ca]

> Gesendet: Mittwoch, 2. September 2015 17:28

> An: kst at kde.org<mailto:kst at kde.org>; Huber, Jörg

> Betreff: Re: [Kst] Trending from daily CSV Files KST-Plot Windows

> Version

>

> Jörg,

>

> Sorry for the slow reply.  I have been very busy with getting a

> payload ready to fly - and will be for another month - so I may not be

> particularly responsive...

> I think we should be able to do what you want using python scripting,

> which we have just recently got working under windows.  Though I also

> suspect that the attempt to do so will produce a fair number of

> opportunities to fix some bugs :-) Regarding file dialog messages:

> does File->Recent Data Files -> Clean up

> non- existent data files fix it?

> We are looking forward to the new subway lines :-)

>

>

> On Wed, Aug 19, 2015 at 10:03 AM, Huber, Jörg

> <Huber.Joerg at herrenknecht.de<mailto:Huber.Joerg at herrenknecht.de>>

>

>

> wrote:

> > Hello Barth,

> >

> >

> >

> > I hope you are doing fine!

> >

> >

> >

> > I have been in contact with you almost one year ago.

> >

> >

> >

> > As I already told you (see correspondence below) - I really like

> > Kst2Plot which is a great tool

> >

> >

> >

> > and we successfully use it for CSV Real Time Trends & Data Analysis.

> >

> >

> >

> >

> >

> > With a normal PC  (8GB Ram, Intel core I5, 2,6GHz, SSD drive)

> > Kst2Plot still manages to open data of

> >

> >

> >

> > 90 days (each day = 1 file, size of ~100MB, 80.000 datasets/rows)

> > which are in total around 7 million datasets!!!

> >

> >

> >

> > If the program starts it will take some while (up to 30minutes), but

> > after data are loaded it still can be operate

> >

> >

> >

> > with adequate response due to the amount of datasets- absolutely

> > impressing!

> >

> >

> >

> >

> >

> > Now my request:

> >

> >

> > The Kst2Plot “list of files” looks like below.

> >

> >

> >

> > …..

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_01.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_02.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_03.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_04.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_05.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_06.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_07.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_08.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_09.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_10.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_11.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_12.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_13.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_14.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_15.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_16.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_17.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_18.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_19.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_20.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_21.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_22.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_23.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_24.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_25.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_26.CSV*

> >

> >

> >

> > *C:\Data\P109C1b\P109C1b_2015_07_27.CSV*

> >

> >

> >

> > ð  *Each day new created CSV-file added here*

> >

> >

> >

> >

> >

> > The datalogger adds each day a new file which will be display in Kst2Plot.

> > This works absolutely great.

> >

> >

> >

> > To limit the period of days Kst2Plot “list of files” is limited to

> > e.g. 30

> > /60 or 90 days.

> >

> >

> >

> > Days older than 30 /60 / 90 days will be automatically deleted from

> > the Kst2Plot “list of files” when a new file is added.

> >

> >

> >

> > If a file is deleted from the list the data shown in plots of

> > Kst2Plot should be also updated and deleted.

> >

> >

> >

> > The problem is that Kst2Plot must be manually restarted to have an

> > updated plot.

> >

> >

> >

> > Is there any other (automated) way to solve this problem?

> >

> >

> >

> > Restarting of the program should be avoided as  this takes too long

> > in case of  ~60-90 files/days listed in Kst2Plot “list of files”.

> >

> >

> >

> > Please be aware that we are using Windows compiled version of Kst2Plot!

> >

> >

> >

> >

> >

> > Another request

> >

> >

> >

> > If Data Wizard is opened fault messages shown below pop up

> >

> >

> >

> > No disc in drive \device\harddisk4\DR4

> >

> >

> >

> > No disc in drive \device\harddisk4\DR3

> >

> >

> >

> > No disc in drive \device\harddisk4\DR2

> >

> >

> >

> > No disc in drive \device\harddisk4\DR1

> >

> >

> >

> >

> >

> > After confirm  “Cancel/Quit/Try again”  4 times for each drive

> > configuration can be done.

> >

> >

> >

> > Is there any way to avoid this messages?

> >

> >

>

> Registergericht / Register Court: Freiburg HRB 390485 Vorsitzender des

> Aufsichtsrates / Chairman of the Supervisory Board:

> Hans-Jörg Vetter

> Vorstand / Board of Management:

> Dr.-Ing. E. h. Martin Herrenknecht, Vorsitzender / Chairman Dipl.-Ing.

> Gebhard Lehmann, Stellvertreter / Vice Chairman Betriebswirt (VWA)

> Kurt Stiefel, CFO Dipl.-Ing. (FH) Günter Richter Dipl.-Ing. (FH)

> Ulrich Schaffhauser Dipl.-Wirtsch.-Ing. Michael Sprang

--

Barth Netterfield

University of Toronto

416-845-0946

Registergericht / Register Court: Freiburg HRB 390485
Vorsitzender des Aufsichtsrates / Chairman of the Supervisory Board: 
Hans-Jörg Vetter
Vorstand / Board of Management: 
Dr.-Ing. E. h. Martin Herrenknecht, Vorsitzender / Chairman
Dipl.-Ing. Gebhard Lehmann, Stellvertreter / Vice Chairman
Betriebswirt (VWA) Kurt Stiefel, CFO
Dipl.-Ing. (FH) Günter Richter
Dipl.-Ing. (FH) Ulrich Schaffhauser
Dipl.-Wirtsch.-Ing. Michael Sprang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kst/attachments/20151014/b8003b45/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.png
Type: image/png
Size: 6153 bytes
Desc: image004.png
URL: <http://mail.kde.org/pipermail/kst/attachments/20151014/b8003b45/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 4905 bytes
Desc: image003.png
URL: <http://mail.kde.org/pipermail/kst/attachments/20151014/b8003b45/attachment-0005.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 87961 bytes
Desc: image002.png
URL: <http://mail.kde.org/pipermail/kst/attachments/20151014/b8003b45/attachment-0006.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.png
Type: image/png
Size: 59897 bytes
Desc: image006.png
URL: <http://mail.kde.org/pipermail/kst/attachments/20151014/b8003b45/attachment-0007.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: HK_logo.jpg
Type: image/jpeg
Size: 28598 bytes
Desc: not available
URL: <http://mail.kde.org/pipermail/kst/attachments/20151014/b8003b45/attachment-0001.jpg>