[Kst] Antw: Re: Importing extremely long files (>32 Bit)

[Nicolas Brisset]

Hi,

Sorry for being quite silent lately. I've changed positions in the middle of a reorganization and I currently have quite a lot to do.
Plus family stuff as well...
I don't know what Barth is up to, but he's apparently very busy as well.

Anyway, I've noticed Ben has been answering questions: thanks a lot!

Regarding the question below, one of the big changes we did recently was to address exactly that issue (RAM limitation, especially under Windows 32 bits but possibly useful in other scenarios as well). If you use the latest version, which is advisable anyway, you can limit the amount of RAM used while reading ASCII data by checking the "Limit buffer size" option under General settings in the ASCII configuration dialog. You can also use threads, which according to your system may give some speedup.

The difference between this approach and the one suggested by Ben is that in this case you end up with all data loaded and available for plots, equations, plugins, statistics, etc.

I hope that helps, otherwise don't hesitate to report your issue with a bit more details.

Best regards,

Nicolas

----- Mail original -----
> De: "Gunter Koenigsmann" <Gunter.Koenigsmann at semikron.com>
> À: kst at kde.org
> Envoyé: Mardi 14 Avril 2015 15:14:14
> Objet: [Kst] Antw: Re:  Importing extremely long files (>32 Bit)
> 
> Hi, Ben,
> 
> Thanks for answering in nearly no time!
> 
> Just loading half of the data works. And is as fast as I am used from
> kst.
> 
> Keep up your good work,
> and kind regards,
> 
>  Gunter.
> 
> 
> Gunter Königsmann
> R&D
> 

> 
> 
> >>> Ben Lewis <benlewis003 at gmail.com> 13.04.2015 10:00 >>>
> Hi Gunter,
> 
> Kst is capable of loading much bigger data files than 3.8GB, so log
> as
> you have enough RAM.
> 
> When using the Data Wizard to load your data file try reading just a
> sub-section of your data. For
> example, in the Data Range section you can specify to start at sample
> 0
> and read to sample 10000 (or
> some other number that does not max out your available RAM).
> 
> When your data is loaded you can step through to the next subsection
> by
> using the forward and back
> arrows. Kst will release the previous data from memory before loading
> the next set of sub-section data.
> 
> I hope this makes sense. If you still have trouble loading your file
> let me know.
> 
> Regards, Ben
> 
> On 13/04/2015 5:36 PM, Gunter Koenigsmann wrote:
> > Dear all,
> >
> > I have a 64-Bit Ubuntu running with KST 2.0.3 and a 64-Bit Windows
> > System with the KST 2.0.8 from the installer for 64-bit systems.
> >
> > On both systems I tried to load the same .csv file that contains
> about
> > 3.8 Gigabytes of comma-separated values. On Windows the program
> > tells
> me
> > that I wanted to read 2350.36 MB - and only had 1525.09 MB
> available.
> > The reported size of free memory might be wrong: My system tells me
> that
> > I have about 1,5 Gigabytes of hard disk Cache , 1,5 Gigabytes of
> memory
> > that is entirely free and 1 Gigabytes of memory that is in use. But
> > I
> am
> > not entirely sure if the windows installer actually installs a
> 64-bit
> > version of the program.
> >
> > On Linux I get the following message instead:
> > OIODevice::read: Called with maxSize < 0
> >
> > Is there any way to fix this?
> >
> > Thanks a lot,
> > and
> > Kind regards,
> >
> >    Gunter.
> >
> > Gunter Königsmann
> > R&D
> >
> > Tel:     +49 911-6559-6025
> > Fax:    +49 911-6559-776747
> >