[Kst] Path to Kst executable

Sat Nov 2 13:40:59 UTC 2013

On 02.11.2013 12:30, Ben Lewis wrote:
> Hi Peter,
>
> I tried opening the 9GB file with the settings you requested and it crashes.

I found a bug when the limit buffer is a 64 value, but a crash when the limit is <4GB?

>
> I can send you the data file if you think it will help with your debugging. Compressed it's about 700MB in size.

Yes, downloading 700 MB is possible, could you upload it somewhere?

I also have a tool which generates test data: build with -Dkst_test=1,
then there is a binary called "asciifilegenerator.exe". With this tool I've generated a 9GB file
and could load it without a crash.

Could you retry your file with the latest 64bit build?
(I also added a progress bar when Kst searches for rows)

Peter

>
> Ben
>
> On 2/11/2013 1:05 AM, Peter Kuemmel wrote:
>> Hi Ben,
>>
>> you information is very usefull!
>>
>> Kst should also support such big files. But I only tested it with files up to 6GB.
>>
>> Does it work when you open the the 9GB file with the 32bit version and you limit
>> the window size to 100MB and only plot one curve?
>>
>> It seems you found a couple of bugs.
>> I will have a look at it later.
>>
>> Thanks,
>> Peter
>>
>>
>>> Gesendet: Freitag, 01. November 2013 um 14:34 Uhr
>>> Von: "Ben Lewis" <egretengineering at gmail.com>
>>> An: kst at kde.org
>>> Betreff: Re: [Kst] Path to Kst executable
>>>
>>> Hi David,
>>>
>>> I'm working with CSV files. The path to these files is sometimes a location on my local hard drive
>>> and sometimes a location on the local network.
>>>
>>> The latest git version has fixed the long delays I was having.
>>>
>>> Having said that, I tried to open a 9Gb CSV file and after a long pause Kst crashed.
>>>
>>> I don't normally have the need to open files this big. I was just trying an extreme example to
>>> exaggerate any delays.
>>>
>>> Steps to reproduce this are:
>>> Data Wizard>Browse to CSV File>Click Configure OR "Update Type"... Crash
>>>
>>> I tried this with the 64 bit version. Clicking on "configure..." is ok, there is no pause and it
>>> doesn't crash. Clicking on "Change Detection" causes it to hang for at least 3 to 4 minutes but it
>>> does recover. Plotting the data fails with the following error:
>>>
>>> "The file could not be read because not enough memory is available. Try setting a file buffer limit
>>> in the configuration options"
>>>
>>> This is interesting because I have 24Gb or RAM, which I thought would have been more than enough for
>>> a 9Gb file.
>>>
>>> I tried opening the file with a "limit buffer size" of 4Gb which failed with the following error:
>>>
>>> "You requested to read in 9088.95MB of data but it seems that you only have approximately 6333.32MB
>>> of usable memory available you cannot load this much data."
>>>
>>> I've also noticed that entering a number greater than 3000MB in the "limit buffer size" value causes
>>> strange behaviour. e.g. If I enter a value of 4000MB select OK and then re-enter the ASCII file
>>> configuration dialogue the number is changed to -96MB.
>>>
>>> I'm not sure how useful all this information is given that plotting a 9Gb file may not be realistic.
>>> But I can provide more information if anybody would like more details.
>>>
>>> Regards, Ben
>>>
>>> On 31/10/2013 11:38 PM, David McMinn wrote:
>>>> Hi,
>>>>
>>>> With regards to symptom 2 what type of data file are you trying to read from and what path are you trying to read it
>>>> from?
>>>>
>>>> I notice that when I change options in the ASCII data configuration dialog it appears as if KST is trying to
>>>> re-parse the file or validate the new settings as I notice a slight pause (maybe a second with a CSV file of a
>>>> couple of hundred MBs).
>>>>
>>>> I don't think it's related to symptom 1 since Nicolas fixed that and I still get the slight pause when changing
>>>> options in the ASCII data configuration dialog.
>>>>
>>>> Cheers
>>>>
>>>> -----Original Message-----
>>>> From: kst-bounces at kde.org [mailto:kst-bounces at kde.org] On Behalf Of Ben Lewis
>>>> Sent: 31 October 2013 11:01
>>>> To: kst at kde.org
>>>> Subject: [Kst] Path to Kst executable
>>>>
>>>> Kst,
>>>>
>>>> I'm having a number of problems with Kst which are possibly related to each other.
>>>>
>>>> Symptom1
>>>> -------------
>>>> When I try to open a recent kst file from the File menu I get the following error:
>>>>
>>>> Error opening document:
>>>> 'C:/Program Files (x86)/Kst 2.0.7/C:/kstFile.kst'
>>>> File could not be opened for reading.
>>>>
>>>> where C:/kstFile.kst' is the path to the kst file I'm trying to open.
>>>>
>>>> After every unsuccessful attempt to load the file the text in the titlebar appends the path to the Kst executable in
>>>> front of the file name. So, for example, after three attempts the text in the title bar will look like this:
>>>>
>>>> 'C:/Program Files (x86)/Kst 2.0.7/C:/Program Files (x86)/Kst 2.0.7/C:/Program Files (x86)/Kst 2.0.7/C:/kstFile.kst'
>>>>
>>>> I also see the path to the Kst executable added to the kst file name in other places like the Data Wizard and the
>>>> Change Data File Dialog box.
>>>>
>>>>
>>>> Symptom 2
>>>> --------------
>>>> Whenever I try to load/change or generally work with a data file the dialogue box freezes for a long
>>>> time (10 - 20 sec).
>>>>
>>>> It's like Kst hangs on every file operation because it can't find the file it's looking for (because
>>>> it has the path to the executable added to it) and eventually it gives up.
>>>>
>>>> Environment
>>>> ----------------
>>>> Kst 2.0.7
>>>> Windows7 x64
>>>>
>>>> What I've tried
>>>> ------------------
>>>> Uninstalling and reinstalling
>>>> Uninstalling 2.0.7 and reinstalling 2.0.6
>>>> Registry clean
>>>>
>>>> Nothing has worked.
>>>>
>>>> I haven't always had this problem but I don't know when it crept in.
>>>>
>>>> Any ideas on how to fix this problem?
>>>>
>>>> Regards, Ben
>>>> _______________________________________________
>>>> Kst mailing list
>>>> Kst at kde.org
>>>> https://mail.kde.org/mailman/listinfo/kst
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Kst mailing list
>>>> Kst at kde.org
>>>> https://mail.kde.org/mailman/listinfo/kst
>>> _______________________________________________
>>> Kst mailing list
>>> Kst at kde.org
>>> https://mail.kde.org/mailman/listinfo/kst
>>>
>> _______________________________________________
>> Kst mailing list
>> Kst at kde.org
>> https://mail.kde.org/mailman/listinfo/kst
>
> _______________________________________________
> Kst mailing list
> Kst at kde.org
> https://mail.kde.org/mailman/listinfo/kst
>