Sextractor deblending

[Hans]

Hi Robert,

Awesome work ! And funny name, though I had expected SourceSolver ;-)

You mentioned 'and maybe even guiding', that is exactly what StarAid
https://staraid.ai/ does, using a local installation of astrometry.net (in that
small device) in real time (10 Hz) to guide. Just letting you know in case you
did not do so already.

Next I think it would be awesome if SexySolver would get a web api frontend
such that it could be used as a drop-in replacement for nova.astrometry.net,
just like what astrometry-api-lite https://github.com/Jusas/astrometry-api-lite
does. Users could then easily host it on a different (and maybe faster)
computer. Being a cross-platform library you made this api addon a lot easier
to implement.

You mentioned that SexySolver is a lot faster than astrometry.net, can you give
some stats of that ? Is it just the bunch of scripts piped together ? I thought
I saw that most of the time was spent in running through the indexes (except
when there are good begin coordinates and arcsec per pixel details, then the
solves I do are about 6 seconds, which is fast to me but still far away from
the <0.1 seconds that staraid takes)

Is (part of) your efforts intended to be pushed upstream to astrometry.net ?

Great work !

-- Hans

Robert Lancaster wrote on 20200414:
> Hi Akarsh,
> 
> Plate solving is a very interesting topic.  It is very complex and yes there are many options, in fact so many that it almost becomes an art more than a science.  There are many packages that can plate solve today, but chief among them is astrometry.net <http://astrometry.net/> and other code based on astrometry.net <http://astrometry.net/>.  There are a couple of parts to plate solving an image:
> 
> 1. Acquire and load the image 
> 2. Find the stars in the image
> 3. Compare the star positions to known star positions to find where it is in the sky.
> 
> So steps 1 and 2 sound easy enough, but they really aren’t as easy as you think.  Astrometry.net <http://astrometry.net/> and over solvers typically rely on other programs to accomplish those tasks and then just focus on step 3.  For astrometry.net <http://astrometry.net/>, if you give it an image, it uses python code, external image libraries and programs like cfitsio, netpbm, libjpeg, and others to load the image and then do the star extraction.  The biggest challenge I have found to getting astrometry.net <http://astrometry.net/> working on MacOS, Windows and other machines is just getting it set up and configured correctly so that it can run with all of these external programs.  On Linux, which astrometry.net <http://astrometry.net/> was designed for, you can just specify those as dependencies and it will just work.  So in February of this year, when I was helping yet another user trying to get python working properly on his MacOs machine, I got frustrated and decided to seek out a different method that wouldn’t require python and so many external programs that are a pain to install.   I found that sextractor could work wonderfully for this purpose and could basically replace python, netpbm, and most of the external code and configuration steps.  So I introduced that into KStars as an alternate option users could choose if they had trouble with astrometry.net’s traditional configuration.  In terms of the options for sextractor that I used, at the time I didn’t worry about most of them since the defaults would probably be fine and I just got it running with the most basic options and settings possible.  I was really shocked when we found out that in fact my alternative solution, not only solved the problem I was trying to address, but actually worked better than the traditional way of using astrometry.net <http://astrometry.net/>.  It solved images in significantly less time.  So there are many options to sextractor, but most of them can be left to the defaults for now.  I am currently exploring the other settings to see what difference they can make.
> 
> So based on my initial success with using Sextractor combined with Astrometry.net <http://astrometry.net/>, I got to thinking that in fact we already were using a library version of Sextractor called SEP in KStars to do a number of tasks and I was wondering if it might be possible to do something similar with astrometry.net <http://astrometry.net/>.  Using these programs as internal libraries would save the user from having to install/configure them, would simplify the installation of KStars, would avoid having to use external program calls to accomplish tasks, etc.  But I knew that if I started trying to do that in KStars, I would get slowed down by all of the other distractions in KStars, so I decided that the best approach would be to take all of the code that is relevant to loading images, extracting them, and solving them into a separate program so that I could just play around and see what I could do with it.  And even if it wasn’t possible to make astrometry.net <http://astrometry.net/> a library, we could just use this program to try to perfect the algorithms and settings for loading, sextracting, and plate solving images to make KStars do it better.   As I started playing with that, I found that it was working pretty well, and I kept increasing the scope of what I was trying to do.  Also, I knew that the reason we can’t do plate solving as easily on windows computers is because of all of the reliance on POSIX code, running external programs, and similar issues.  But as I explored the astrometry code, I found that quite a lot of the code could just be eliminated if we used sextractor to get the stars from the image and just focus astrometry.net <http://astrometry.net/> on solving only.  So I got even more bold and after about a month of work in my spare time, I managed to get it working on windows!   So now, after about 2 months of work, I have created a package that works on Mac, Windows, and Linux, that can be run as an internal library in KStars, that makes no external temporary files, doesn’t run any external programs, and can be used to thoroughly test all of those settings so that we can perfect them for KStars.  I am calling it SexySolver because it combines the library version of Sextractor and the astrometry.net <http://astrometry.net/> solver engine to make it work.
> 
> So now my plan is to put some finishing touches on it and send out my tester program so that people can start testing this on images, playing with settings so that we can perfect those settings, fix problems etc.  After we get it perfected, then we can plop the internal libraries into KStars and use that for star extraction, HFR calculations, plate solving, focusing, and maybe even guiding
> 
> Some Useful Links:
> 
> Astrometry.net <http://astrometry.net/> Documentation: http://astrometry.net/doc/readme.html <http://astrometry.net/doc/readme.html>
> Sextractor/SEP Documentation: https://sep.readthedocs.io/en/v1.0.x/ <https://sep.readthedocs.io/en/v1.0.x/>
> Sextractor Manual: https://www.astromatic.net/pubsvn/software/sextractor/trunk/doc/sextractor.pdf <https://www.astromatic.net/pubsvn/software/sextractor/trunk/doc/sextractor.pdf>
> 
> My new SexySolver tester and Library: https://github.com/rlancaste/sexysolver-tester <https://github.com/rlancaste/sexysolver-tester>
> 
> Thanks,
> 
> Rob
> 
> 
> 
> > On Apr 14, 2020, at 4:44 AM, Akarsh Simha <akarsh at utexas.edu> wrote:
> > 
> > Sorry this is going slightly OT, but I'm a newbie to all of this, and am trying to get a sense of how to plate solve. Would you be able to point me to some documentation / information about all of these options to sextractor, what SexySolver is, etc.?
> > 
> > Regards
> > Akarsh