KST Speed

[Erik Li]

Hi all,

What makes KST so fast? I've tried to look at the source but I'm having a
hard time getting a high-level overview of what is going on. I see
interpolation being defined in libkst/vector.cpp and can see some places
where it's called in libkstmath/curve.cpp, but the way that NS (num
samples) is being set would suggest to me that the interpolation is in fact
returning the raw data. My naive thought of how this should work is that
you essentially subsample the data to plot a smaller portion of the data,
but the accuracy with which kst plots things at different scales somewhat
belies this.

I've been super impressed by the application and it's responsiveness. For
reference, I am trying to analyze data from electrophysiological
experiments, which for my workflow means time series datasets of around 1.5
to 15 million datapoints. Kst handles this like a champ with quick
panning/zooming. Even more impressive for me is that it maintains good
temporal resolution for the action potentials and doesn't cut them short
even when zoomed out. (The signals that I look at basically run from -60 to
-50 mV most of the time and then periodically "spike" to 20 mV for 1 ms at
a time. Those spikes are action potentials and the exact timing is pretty
important to my analyses)

We have an analysis workflow that uses R or python, but the plotting speed
takes sometimes tens of seconds to plot the data. I am trying to make a
small application that will allow reasonable display of the data and be
responsive to changes in analysis parameters so that you can actually
"play" with the data in an exploratory fashion.

I see two paths forward for me:
  - Implement my analysis routines in kst using the plugin framework
    - in this case I would want to capture mouse events on the plot also.
Are there bindings for mouse/keyboard events on the plot?
  - Implement parts of the KST plotting algorithm for curves in R or python

I'm happy to try to move forwards in whatever way is going to make the most
sense, but in either case I would be really interested in getting a sense
of the algorithm that Kst is using to plot curves because it seems like
there is some quite sophisticated stuff going on here. I would have thought
fast plotting would be a broadly solved problem, but really in the open
source space Kst is the only solution I've found that works at this scale.
(Maybe also gnuplot, but I haven't explored it very carefully yet).

Thank you in advance for any insight or pointers on places in the source
that I should be looking at! And if this is explained somewhere on the
website or in a blog post sorry for my cluelessness.

-Erik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kst/attachments/20180505/c60bf126/attachment.html>