Sextractor deblending

Wed Apr 8 07:16:53 BST 2020

Hello Hy,

Robert developed the SexySolver tester here:
https://github.com/rlancaste/sexysolver-tester

Of course, Robert can explain more but I believe the sexractor part is
complete for Windows, MacOS, and Linux.

Internal solver is also almost done. Seems there is a problem with loading
some index files on Windows OS. I tried to look into it but failed to find
out why. It needs more detective work. IIRC, it's 4218 index that fails to
load all FITS extensions. Here is what Robert told me about the issue the
other day:

> so no, 4218 loads only 9 extensions on Windows
> it fails after lucky number 9
> On Mac/Linux, that same file loads 14 extensions
> so I was trying to figure out what happens between extension 9 and 10.  It
> tries to continue loading extensions but encounters all those nulls
> Here is a piece of the log file that sent me to investigate this in the
> first place.  THIS ONE WORKS and is from my Mac:
> Trying to add index "/Users/rlancaste/Library/Application
> Support/Astrometry/index-4218.fits".
> Index name "/Users/rlancaste/Library/Application
> Support/Astrometry/index-4218.fits" is readable; assuming singe file.
> Index name "/Users/rlancaste/Library/Application
> Support/Astrometry/index-4218.fits" is readable; assuming singe file.
> Index name "/Users/rlancaste/Library/Application
> Support/Astrometry/index-4218.fits" is readable; assuming singe file.
> N ext: 14
> kdtree_fits_open() took 0.156 ms
> kdtree_fits_contains_tree() took 0.276 ms
> searched 14 extensions in file /Users/rlancaste/Library/Application
> Support/Astrometry/index-4218.fits but didn't find a table with a column
> "kdtree_perm_stars".
> searched 14 extensions in file /Users/rlancaste/Library/Application
> Support/Astrometry/index-4218.fits but didn't find a table with a column
> "kdtree_bb_stars".
> searched 14 extensions in file /Users/rlancaste/Library/Application
> Support/Astrometry/index-4218.fits but didn't find a table with a column
> "kdtree_splitdim_stars".
> kdtree_fits_read_tree() took 0.28 ms
> reading chunks took 0.01 ms
> N ext: 14
> N ext: 14
> searched 14 extensions in file /Users/rlancaste/Library/Application
> Support/Astrometry/index-4218.fits but didn't find a table with a column
> "kdtree_perm_codes".
> searched 14 extensions in file /Users/rlancaste/Library/Application
> Support/Astrometry/index-4218.fits but didn't find a table with a column
> "kdtree_bb_codes".
> searched 14 extensions in file /Users/rlancaste/Library/Application
> Support/Astrometry/index-4218.fits but didn't find a table with a column
> "kdtree_splitdim_codes".
> Index scale: [1000, 1400] arcmin, [60000, 84000] arcsec
> Index has 3072 quads and 1920 stars
> index_load("/Users/rlancaste/Library/Application
> Support/Astrometry/index-4218.fits") took 1.54698 ms
>

*Now on Windows:*

> THIS ONE FAILS AND IS FROM WINDOWS:
> Trying to add index "C:/Astrometry/index-4218.fits".
> Index name "C:/Astrometry/index-4218.fits" is readable; assuming singe
> file.
> Index name "C:/Astrometry/index-4218.fits" is readable; assuming singe
> file.
> Index name "C:/Astrometry/index-4218.fits" is readable; assuming singe
> file.
> N ext: 9
> kdtree_fits_open() took 0 ms
> kdtree_fits_contains_tree() took 1 ms
> searched 9 extensions in file C:/Astrometry/index-4218.fits but didn't
> find a table with a column "kdtree_perm_stars".
> searched 9 extensions in file C:/Astrometry/index-4218.fits but didn't
> find a table with a column "kdtree_bb_stars".
> searched 9 extensions in file C:/Astrometry/index-4218.fits but didn't
> find a table with a column "kdtree_split_stars".
> searched 9 extensions in file C:/Astrometry/index-4218.fits but didn't
> find a table with a column "kdtree_splitdim_stars".
> searched 9 extensions in file C:/Astrometry/index-4218.fits but didn't
> find a table with a column "kdtree_data_stars".
> searched 9 extensions in file C:/Astrometry/index-4218.fits but didn't
> find a table with a column "kdtree_range_stars".
> kdtree_fits_read_tree() took 2 ms
> index_load("C:/Astrometry/index-4218.fits") took 4.00007 ms
> Failed to add index "C:/Astrometry/index-4218.fits".
> Note that the reason index 4218 did not get loaded properly is because it
> only found 9 extensions, when there are 14 it should load
> 4118 and 4109 fail for the same reason
> whatever that reason is

Once the above issues are resolved, we can probably start integrating both
the internal solver & sextractor into KStars and mass-test it with all
kinds of conditions.

So if you test the SexySolver above, you can see all the options that, I
believe, we ought to present in KStars. I've been thinking that given the
power & versatility of sextrator, we should probably plan on setting it as
the default star-detection algorithm across KStars. Moreover, I believe we
should have general settings and also the ability to apply settings per
camera basis. Perhaps this can follow some sort of inheritance scheme where
the camera-specific sextractor setting overrides the general settings when
the user starts selecting specific options to modify.

Finally, I believe we should send all development emails to KStars
development mailing list to get the most exposure from all the developers &
users who are subscribed to the mailing list.
--
Best Regards,
Jasem Mutlaq

On Wed, Apr 8, 2020 at 7:23 AM Hy Murveit <murveit at gmail.com> wrote:

> Pit,
>
> I seem to remember you being a "Sextractor Wizard" ;)
>
> I've got a question. Alfred Herrhausen, from the Indi Forum, sent me an
> image which showed the SEP Extractor in Ekos detecting many double-stars as
> stars, with, of course, corresponding larger-than-necessary HFRs.
>
> [image: Screen Shot 2020-04-07 at 9.17.54 PM.png]
>
> I looked into this, and found that in our implementation, we're disabling
> the deblending by setting the deblending params as follows:
>
> DEBLEND_NTHRESH 32
> DEBLEND_MINCONT 1.0
>
>
> The first I guess is the default, but the latter one disables all
> deblending.
>
> On his image (a full-exposure sub, 300s RGB) I set DEBLEND_MINCONT to 0.01
> and almost all of the doubles went away. I have not tried this on many
> other images.
>
> I was curious what your recommendation would be for this for KStars (if we
> were to have only one value) or if it is necessary to have several. E.g. we
> want to detect starts for captures with:
>
>    - short exposures (e.g. a few seconds on an L, R, G, B, Ha, OIII, SII
>    filter, or color camera) for autofocus
>    - short exposures (L or RGB) for plate solving
>    - long sub exposures with all possible filters for tracking HFR on the
>    capture sequence.
>
> Of course, the image might contain nebulae, galaxies and/or star
> fields, and be wide field or long focal lengths.
>
> What do you think? Could one value be decent? We've survived so far with
> 1.0. Would something else be an improvement?
>
> Thanks,
> Hy
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kstars-devel/attachments/20200408/2333ec6d/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screen Shot 2020-04-07 at 9.17.54 PM.png
Type: image/png
Size: 4048648 bytes
Desc: not available
URL: <http://mail.kde.org/pipermail/kstars-devel/attachments/20200408/2333ec6d/attachment-0001.png>