[Kst] Trending from daily CSV Files KST-Plot Windows Version

Wed Sep 30 14:05:52 UTC 2015

Sorry for the slow reply.  I'm finally back.

The intended workflow (and the mode we operate in) is to never remove files from 
the beginning of the 'list of files' and to instead operate in 'count from end' 
mode instead of 0 to end mode.  The number of frames listed in 'count from 
end' should be whatever is needed for the past 30/60/90 days.  This should be 
fast.

The data will scroll by as new data comes.

Does this work for you?

cbn

On September 22, 2015 10:33:59 AM Huber, Jörg wrote:

> Hello Barth,
>  
> Thanks a lot for your answers and help!
> I was on holiday, so sorry for the delay in my reply.
>  
> I did some more tests and I must correct my statement:
> If data vector(s) is/are configured "read from start = 0 to end"
>                 => 1. plots are updated if one or more files are deleted in
> KstPlot “list of files” e.g. 1st entry = oldest file of 30 files is deleted
> at the beginning of "list of files" 
>                => 2. the same happens if one file is add to "list of files"
>  
> “list of files”
> 
>  
>  
>
> The difference between case 1 + 2 is as described below:
>  
> case 2: plot update is fast as additional data are added to existing plot
> (<10sec) which is perfect! 
> case 1: plot update is slow as additional data are delete from existing plot
> Depending on how many files in “list of files” and data points (here:
> 30files à 80.000 data points) Window of KstPlot is frozen as data curve /
> plot must be updated or re-arranged from the beginning. This takes in my
> case about 10/20/30 min with 30/60/90 files.
>                 At this time user interface of KstPlot and online charts
> will be frozen and no longer actualized.               
> Is there a possibility to solve this problem? Maybe using additional
> “task/background worker” for data vector update? I’m not familiar with
> phyton scripting but please let me know, if and how the problem could be
> managed by phyton script in a later release of KstPlot. 
>  
> Regarding:  file dialog messages: it seems that "clean up non existing
> files" fixed it! 
> Thanks a lot
>  
> Best regards
> Jörg
>  
>  
> 
> Best regards
> 
> Jörg Huber
> Graduate Engineer
> electrical engineering
> ____________________________________________________
> 
> Herrenknecht AG
> Schlehenweg 2
> 77963 Schwanau-Allmannsweier
> Germany
> (+49 (7824) 302 5440
> 7+49 (7824) 302 2940
> 
> mailto:huber.joerg at herrenknecht.de
> http://www.herrenknecht.de
>  
> 
>  
>  
>  
> -----Ursprüngliche Nachricht-----
> Von: Barth Netterfield [mailto:barth.netterfield at utoronto.ca]
> Gesendet: Mittwoch, 2. September 2015 17:28
> An: kst at kde.org; Huber, Jörg
> Betreff: Re: [Kst] Trending from daily CSV Files KST-Plot Windows Version
>  
> Jörg,
>  
> Sorry for the slow reply.  I have been very busy with getting a payload
> ready to fly - and will be for another month - so I may not be particularly
> responsive... 
> I think we should be able to do what you want using python scripting, which
> we have just recently got working under windows.  Though I also suspect
> that the attempt to do so will produce a fair number of opportunities to
> fix some bugs :-) 
> Regarding file dialog messages: does File->Recent Data Files -> Clean up
> non- existent data files fix it? 
> We are looking forward to the new subway lines :-)
>  
>  
> On Wed, Aug 19, 2015 at 10:03 AM, Huber, Jörg <Huber.Joerg at herrenknecht.de>
>  
> 
> wrote:
> > Hello Barth,
> >
> > 
> >
> > I hope you are doing fine!
> >
> > 
> >
> > I have been in contact with you almost one year ago.
> >
> > 
> >
> > As I already told you (see correspondence below) - I really like
> > Kst2Plot which is a great tool
> >
> > 
> >
> > and we successfully use it for CSV Real Time Trends & Data Analysis.
> >
> > 
> > 
> > 
> >
> > With a normal PC  (8GB Ram, Intel core I5, 2,6GHz, SSD drive) Kst2Plot
> > still manages to open data of
> >
> > 
> >
> > 90 days (each day = 1 file, size of ~100MB, 80.000 datasets/rows)
> > which are in total around 7 million datasets!!!
> >
> > 
> >
> > If the program starts it will take some while (up to 30minutes), but
> > after data are loaded it still can be operate
> >
> > 
> >
> > with adequate response due to the amount of datasets- absolutely
> > impressing!
> >
> > 
> > 
> > 
> >
> > Now my request:
> > 
> >
> > The Kst2Plot “list of files” looks like below.
> >
> > 
> >
> > …..
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_01.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_02.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_03.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_04.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_05.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_06.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_07.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_08.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_09.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_10.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_11.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_12.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_13.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_14.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_15.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_16.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_17.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_18.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_19.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_20.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_21.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_22.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_23.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_24.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_25.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_26.CSV*
> >
> > 
> >
> > *C:\Data\P109C1b\P109C1b_2015_07_27.CSV*
> >
> > 
> >
> > ð  *Each day new created CSV-file added here*
> >
> > 
> > 
> > 
> >
> > The datalogger adds each day a new file which will be display in Kst2Plot.
> > This works absolutely great.
> >
> > 
> >
> > To limit the period of days Kst2Plot “list of files” is limited to
> > e.g. 30
> > /60 or 90 days.
> >
> > 
> >
> > Days older than 30 /60 / 90 days will be automatically deleted from
> > the Kst2Plot “list of files” when a new file is added.
> >
> > 
> >
> > If a file is deleted from the list the data shown in plots of Kst2Plot
> > should be also updated and deleted.
> >
> > 
> >
> > The problem is that Kst2Plot must be manually restarted to have an
> > updated plot.
> >
> > 
> >
> > Is there any other (automated) way to solve this problem?
> >
> > 
> >
> > Restarting of the program should be avoided as  this takes too long in
> > case of  ~60-90 files/days listed in Kst2Plot “list of files”.
> >
> > 
> >
> > Please be aware that we are using Windows compiled version of Kst2Plot!
> >
> > 
> > 
> > 
> >
> > Another request
> >
> > 
> >
> > If Data Wizard is opened fault messages shown below pop up
> >
> > 
> >
> > No disc in drive \device\harddisk4\DR4
> >
> > 
> >
> > No disc in drive \device\harddisk4\DR3
> >
> > 
> >
> > No disc in drive \device\harddisk4\DR2
> >
> > 
> >
> > No disc in drive \device\harddisk4\DR1
> >
> > 
> > 
> > 
> >
> > After confirm  “Cancel/Quit/Try again”  4 times for each drive
> > configuration can be done.
> >
> > 
> >
> > Is there any way to avoid this messages?
> >
> > 
> 
> Registergericht / Register Court: Freiburg HRB 390485
> Vorsitzender des Aufsichtsrates / Chairman of the Supervisory Board:
> Hans-Jörg Vetter
> Vorstand / Board of Management:
> Dr.-Ing. E. h. Martin Herrenknecht, Vorsitzender / Chairman
> Dipl.-Ing. Gebhard Lehmann, Stellvertreter / Vice Chairman
> Betriebswirt (VWA) Kurt Stiefel, CFO
> Dipl.-Ing. (FH) Günter Richter
> Dipl.-Ing. (FH) Ulrich Schaffhauser
> Dipl.-Wirtsch.-Ing. Michael Sprang

-- 
Barth Netterfield
University of Toronto
416-845-0946