AW: Read 2 Sample per... Fails.

[Oltersdorf, Thore]

Hi Dan,

have your tried pre-process your data? If you are on your way in *NIX systems you could easily use the program "cut" to get all fields out that you are interested in:

cut -d'XXX' -fYYY putyourfilename.here

Substitute 'XXX' and YYY by your field delimiter and your field range of interest. This could be a single field by a single number or a field range e.g. 2-5 and you also could use a set of fields by 2,3-6 or similar. The 32M lines would stay the same. 

Or using "awk" to get a range of lines but keeping field numbers the same:

awk 'NR==122,NR==129' putyourfilename.here

Substitute 122 and 129 to the range your are interested in.

Before doing this sometimes it is necessary that a file need to be converted due to the different handling of EOL (end-of-lines) in *NIX and Windows systems by using the program "dos2unix".

Kind regards,
Thore

-----Ursprüngliche Nachricht-----
Von: Kst <kst-bounces at kde.org> Im Auftrag von Dan Kemppainen
Gesendet: Dienstag, 7. März 2023 21:40
An: kst at kde.org
Betreff: Read 2 Sample per... Fails.

Hi All,

Trying to read a large data file. This is an ASCII csv file. Need to run FFT on the data. File is too large to FFT the whole thing, so trying to skip samples to try FFT the data.

The problem is, the Read 1 Sample per (x) frames crashes. It either throws and error, or just hangs and crashes.

The error reads:
Error while reading ASCII file.
The file was read only partially.
Try without threads or use a different file buffer limit when using threads for reading.

It happens in x32 and x64 versions. Both running windows.

ASCII file is 32M lines, four columns. Reads and displays OK, just crashes when trying to reduce samples.

Any ideas?

Thanks!
Dan