KST Speed

Erik Li erikli at drexel.edu
Thu May 10 17:46:04 UTC 2018


Dear Barth,

Thanks for the explanation - it really is a clever method! I appreciate you
taking the time to break it down and for all the time/effort that has been
put into this piece of software.

Regarding pykst, I had taken a look at it and it seems like a great option
for plotting but (unless I missed it) doesn't support reporting of
mouse/keyboard events, so it would be hard for me to implement some of the
interactive analysis features I was hoping to include. It will likely be a
great option for my personal analysis, but for some of my lab members less
direct interaction with the script = better. It may still be a good option
depending on where I decide to make the tradeoff between ease of
implementation/ease of use.

Thanks again,
Erik

On Wed, May 9, 2018 at 10:47 PM, Barth Netterfield <
barth.netterfield at gmail.com> wrote:

> Hi, Erik,
>
> Sorry for the slow reply - KST may be fast, but apparently I'm not always.
>
> If the X axis vector is monotonically increasing, then the curve drawing
> is optimized:
>   Noting that the X resolution is finite (especially compared to a very
> large vector).
>   Noting that one only needs to draw a line from the minimum value in a
> column of pixels to the maximum value in the row of pixels in order to not
> miss anything.
>
> and then doing the obvious thing.  Since looping through points and doing
> compares is far faster than drawing lots and lots of lines, the drawing
> gets way way faster.
>
> As to your problem:  You could consider using pykst.  You would do all of
> your math and manipulation in python, as you do now, then plot it in kst.
>
> You can plot numpy arrays in kst.  It takes about 0.14s to copy a
> 15,000,000 point numpy array into a running kst session and plot it.  You
> should be able to overwrite and re-draw the vector in kst at will, but
> still have a fully live kst session to zoom and scroll, etc.  pykst has
> full control of most of the kst session, so this should actually work
> pretty well I think.
>
> Take a look at https://na01.safelinks.protect
> ion.outlook.com/?url=https%3A%2F%2Fkst-plot.kde.org%2Fpykst%
> 2F&data=02%7C01%7Cezl24%40drexel.edu%7C5e8786f1e25d483
> aa2e408d5b6206d03%7C3664e6fa47bd45a696708c4f080f8ca6%7C0%
> 7C0%7C636615172756865682&sdata=cVFXn1pFugyRROMkQJE9kihV
> EOzIYUCfQziKaA6sBIo%3D&reserved=0
>
> cbn
>
>
>
>
>
>
> On 2018-05-05 01:12 AM, Erik Li wrote:
>
>> Hi all,
>>
>> What makes KST so fast? I've tried to look at the source but I'm having a
>> hard time getting a high-level overview of what is going on. I see
>> interpolation being defined in libkst/vector.cpp and can see some places
>> where it's called in libkstmath/curve.cpp, but the way that NS (num
>> samples) is being set would suggest to me that the interpolation is in fact
>> returning the raw data. My naive thought of how this should work is that
>> you essentially subsample the data to plot a smaller portion of the data,
>> but the accuracy with which kst plots things at different scales somewhat
>> belies this.
>>
>> I've been super impressed by the application and it's responsiveness. For
>> reference, I am trying to analyze data from electrophysiological
>> experiments, which for my workflow means time series datasets of around 1.5
>> to 15 million datapoints. Kst handles this like a champ with quick
>> panning/zooming. Even more impressive for me is that it maintains good
>> temporal resolution for the action potentials and doesn't cut them short
>> even when zoomed out. (The signals that I look at basically run from -60 to
>> -50 mV most of the time and then periodically "spike" to 20 mV for 1 ms at
>> a time. Those spikes are action potentials and the exact timing is pretty
>> important to my analyses)
>>
>> We have an analysis workflow that uses R or python, but the plotting
>> speed takes sometimes tens of seconds to plot the data. I am trying to make
>> a small application that will allow reasonable display of the data and be
>> responsive to changes in analysis parameters so that you can actually
>> "play" with the data in an exploratory fashion.
>>
>> I see two paths forward for me:
>>   - Implement my analysis routines in kst using the plugin framework
>>     - in this case I would want to capture mouse events on the plot also.
>> Are there bindings for mouse/keyboard events on the plot?
>>   - Implement parts of the KST plotting algorithm for curves in R or
>> python
>>
>> I'm happy to try to move forwards in whatever way is going to make the
>> most sense, but in either case I would be really interested in getting a
>> sense of the algorithm that Kst is using to plot curves because it seems
>> like there is some quite sophisticated stuff going on here. I would have
>> thought fast plotting would be a broadly solved problem, but really in the
>> open source space Kst is the only solution I've found that works at this
>> scale. (Maybe also gnuplot, but I haven't explored it very carefully yet).
>>
>> Thank you in advance for any insight or pointers on places in the source
>> that I should be looking at! And if this is explained somewhere on the
>> website or in a blog post sorry for my cluelessness.
>>
>> -Erik
>>
>
> --
> C. Barth Netterfield
> 416-845-0946
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kst/attachments/20180510/e9d555bf/attachment-0001.html>


More information about the Kst mailing list