Akshay--a New Volunteer

[Jasem Mutlaq]

Hello Akshay,

Just wanted to follow up on the ML Guider concept. Do you think it's
doable? If yes, what do we need to do next? What kind of training data is
required?

--
Best Regards,
Jasem Mutlaq



On Tue, Mar 18, 2025 at 10:05 AM Akshay Subramaniam <akshaysubr at gmail.com>
wrote:

> Regarding the ML guider concept, all of these ideas sound good. I don’t
> quite have a handle on what the “ground truth” data would be though. If we
> just use guiding pulses from the logs, we can only ever match the
> performance of existing algorithms, right? I think it might be prudent to
> do a bit of a literature survey on ML based control algorithms generally.
> One specific paper comes to mind where they trained an ML control algorithm
> that can hover a helicopter upside down but I can’t remember the title of
> the paper off the top of my head.
>
> I agree that the model must be specific to a users setup, but I also feel
> like having a base model that has learnt fundamental guiding
> characteristics would be very useful so a user can fine tune that on their
> specific setup rather than train an all new model from scratch. Anyway,
> I’ll do some literature search over the next week or so to have a more
> informed opinion on this topic.
>
> Thanks,
> Akshay
>
> On Sun, Mar 16, 2025 at 11:50 PM John Evans <john.e.evans.email at gmail.com>
> wrote:
>
>> My 2c on the ML/AI guider concept. It seems a good choice as we already
>> have guide logs that detail the pulses sent to the mount so there is a
>> ready source of data for training although perhaps a first step would be to
>> review these logs with a view to their suitability for AI.
>>
>> The procedural approach currently is 2 fold:
>> 1. Standard Correction. Calculate the deviation and from the calibration
>> send a correction pulse. Then repeat.
>> 2. Pre-emptive Correction. Based on periodic error, calculate repeatable
>> deviation and send these corrections at the appropriate time to minimise
>> deviation in the first place. This is PEC / PPEC / GPG.
>>
>> It kind of feels like a good opportunity for an AI algorithm although as
>> others have suggested maybe not as a first step. Maybe however it would be
>> OK to start thinking about how we could go about this in the future and see
>> what prerequisites we'd need (e.g. additional data in the guide logs)?
>>
>> On Mon, 17 Mar 2025 at 03:52, Jasem Mutlaq <mutlaqja at ikarustech.com>
>> wrote:
>>
>>> Hello Akshay,
>>>
>>> Welcome abroad! I find that Machine Learning powered guiding is indeed
>>> very exciting but as you rightfully indicated, very challenging for a first
>>> time task! You are indeed correct, we used AI+RAG to answer FAQs and other
>>> knowledge based articles related to StellarMate.
>>>
>>> The binning statistics sound good. There are also many KStars issues
>>> reported here that would be of interest:
>>> https://invent.kde.org/education/kstars/-/issues
>>>
>>> Regarding the ML Guiding, I'm not sure if this is something we can
>>> crowdsource per se. My intuition is that it should learn on the user's own
>>> setup due to many contributing factors in seeing, mount, camera, and optics
>>> that can vastly differ from one user or another. It would be possible to
>>> start "training" after a guiding session is over for example, or while it
>>> is running, or in a special "train mode". At any rate, I am by no means an
>>> expert on this, so feel free to suggest a better model to approach this
>>> issue!
>>>
>>> --
>>> Best Regards,
>>> Jasem Mutlaq
>>>
>>>
>>>
>>> On Mon, Mar 17, 2025 at 4:52 AM Akshay Subramaniam <akshaysubr at gmail.com>
>>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Firstly, I want to say I really appreciate all of your work developing
>>>> kstars and INDI and for putting out a ton of education material as well.
>>>> And thanks for all the suggestions on how to get started. Just as a
>>>> warning, I've never contributed to a full stack application and have no
>>>> clue how frontend development works so I might be very slow to implement
>>>> things in kstars 😅
>>>>
>>>> The AI based guiding algorithm sounds fun but also daunting as a first
>>>> effort. I assume the biggest challenge would be inferencing the model in
>>>> real time faster than a single guide exposure at the very least. Maybe the
>>>> first step to tackle that would be to add a harness that outputs a bunch of
>>>> guiding data along with all the relevant mount parameters. That way, we can
>>>> crowd source data from people that would be willing to share. That would be
>>>> necessary to train a base model which folks can then finetune that model
>>>> for their specific mount either as an offline step or on the fly. I'd also
>>>> imagine there would need to be some guardrails around the predicted guiding
>>>> pulses just so things don't go off the rails because of some instability
>>>> mode of the model. Another idea is a simple LLM+RAG model that a kstars
>>>> user can chat with to answer simple questions like "What should my meridian
>>>> flip settings be for an OnStep based mount?". I believe Jasem and the
>>>> Stellarmate folks already have something similar and it can even take
>>>> actions based on natural language commands!
>>>>
>>>> The binning conditional statistics enhancement seems like a simpler
>>>> starting point so I can maybe start with that.
>>>>
>>>> Thanks,
>>>> Akshay
>>>>
>>>> On Sun, Mar 16, 2025 at 6:10 PM Hy Murveit <murveit at gmail.com> wrote:
>>>>
>>>>> Akshay,
>>>>>
>>>>> and I'll throw one in too, related to John's suggestion. How about a
>>>>> adaptive guider based on machine learning? E.g. it decides on the
>>>>> correction pulses based on how well recent pulses have corrected the guide
>>>>> error--so presumably learns about periodic error and backlash. I suppose it
>>>>> could be pre-trained and/or adapting on the fly, or more likely both. I'd
>>>>> be happy to collaborate on that.
>>>>>
>>>>> Hy
>>>>>
>>>>>
>>>>> On Sun, Mar 16, 2025 at 1:28 PM John Evans <
>>>>> john.e.evans.email at gmail.com> wrote:
>>>>>
>>>>>> Hi Akshay,
>>>>>>
>>>>>> Welcome to the project, great to have you on board!
>>>>>>
>>>>>> I had a quick look at the links Hy enclosed and see you’re working on
>>>>>> AI related topics. I’m sure there are some interesting things in this area
>>>>>> that could be brought to KStars. I personally know very little about this
>>>>>> field but would be very interested to collaborate if you think it would be
>>>>>> appropriate.
>>>>>>
>>>>>> Regards,
>>>>>> John.
>>>>>>
>>>>>> On 16 Mar 2025, at 18:53, Hy Murveit <murveit at gmail.com> wrote:
>>>>>>
>>>>>> 
>>>>>> Folks,
>>>>>>
>>>>>> I bumped into Akshay (cc'd) today at a local astro-club function.
>>>>>>
>>>>>> Akshay is an aeronautical engineer at NVidia, with a PhD in physics
>>>>>> from Stanford
>>>>>> and a KStars user, who's interested in contributing to our project.
>>>>>> If you have any suggestions on projects which might help start to
>>>>>> contribute, please feel free to reach out to him at
>>>>>> akshaysubr at gmail.com
>>>>>>
>>>>>> https://developer.nvidia.com/blog/author/akshay-subramaniam/
>>>>>> https://scholar.google.com/citations?user=hNhELoYAAAAJ&hl=en
>>>>>>
>>>>>> Hy
>>>>>>
>>>>>> Hey Hy,
>>>>>>
>>>>>> Nice meeting you today and I really enjoyed our chat. You mentioned a
>>>>>> kstars framing workflow video you had made, I’d love to watch that
>>>>>> incorporate that into my process.
>>>>>>
>>>>>> Also, I’m enthusiastic about contributing to kstars/ekos. If anything
>>>>>> specific comes up, let me know. Maybe something simple to start with so I
>>>>>> get a bit more familiar with the kstars development process before trying
>>>>>> to tackle anything more complicated. I was thinking adding an offset
>>>>>> between plate solving and the mount location might be a nice easy one so
>>>>>> one could use a guide camera and finder scope for plate solving with the
>>>>>> main scope used for visual.
>>>>>>
>>>>>> Thanks,
>>>>>> Akshay
>>>>>>
>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kstars-devel/attachments/20250325/8af109ac/attachment-0001.htm>