Artificial Horizon

Thu Apr 1 08:42:04 BST 2021

Hello Hy, 

This behaviour is the expected one: a job that goes through its altitude restriction will be rescheduled instead of being aborted.

I completely utterly totally agree that this has to be improved to fit other jobs in the current night session. But. 

The original design was preserving the order of the jobs so that the order of the plan created by the end-user would be followed strictly.

From there, an automatic reordering of jobs based on altitude was (re-)attached as a persistent option and as a helper in the UI. Still once the Scheduler started, we wouldn't change a comma. 

From there, it was observed that a plan required consistent statistics about the duration of each step, and (a lot of) fault tolerance and robustness improvements. We worked on this quite a bit. 

From there, Wolfgang proposed three super management schemes for failing jobs to mitigate runtime. The concept of a failing job was discussed at that time, and a job going through its altitude restriction was not considered a failure. 

From there, it was obvious that it was not possible to further change the Scheduler leviathan without introducing regressions to existing features, and that a full test coverage was required. Testing the Scheduler required testing all other modules first. And we did that. 

So today we're here: we have tests for most modules, but not yet for all Scheduler features. Those tests are not stable enough (I'm taking care of this), but we are in a far better situation than before. We absolutely require Scheduler tests before we introduce new features, and I will probably vote against any change that is not covered and tested.

As a side note, I am really impressed by the work we did on this part, so I would like to once again congratulate everyone that took part in it. 

eric.dejouhanet at gmail.com - https://astronomy.dejouha.net

	  Message original  	

De: murveit at gmail.com
Envoyé: 1 avril 2021 08:45
À: eric.dejouhanet at gmail.com
Répondre à: hy at murveit.com
Cc: hy at murveit.com; kstars-devel at kde.org
Objet: Re: Artificial Horizon

I guess in tonight's testing one thing worked and the other didn't go as I'd hoped.
The first job ran until it hit the artificial horizon constraint on its way west/down and got rescheduled for the next day. Good!
The 2nd job could have started up in an hour or two after the first job was stopped. However, the scheduler scheduled it for the next night instead of in a couple hours. I stopped the scheduler, removed the first job, restarted the scheduler, and then the scheduler did the right thing (scheduling it for when it would be above the artificial horizon tonight).
Bottom line, the scheduler is respecting the artificial horizon constraints, but its logic doesn't allow the 2nd job to run for some reason.

Is there a known complication, why the scheduler wouldn't schedule a 2nd job if the first job got suspended because of altitude?

I'll look at the code, but just in case this is a well known complication, here are some log snippets. It's running HEAD plus my MR.

Hy

Original schedule:
M1 starting at 31/03 21:47, completing at 01/04 07:44." (it thinks the first job will take 10 hours--doesn't take altitude into account)
M5 starting at 02/04 00:37, completing at 02/04 08:23." (though it could start around 0:30 on 01/04, it starts the next night because of the first job.

[2021-03-31T21:49:46.835 PDT INFO ][ org.kde.kstars.ekos.scheduler] - Scheduler is starting...
[2021-03-31T21:49:46.837 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Scheduler started."
[2021-03-31T21:49:46.844 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - Scheduler started.
[2021-03-31T21:49:47.904 PDT INFO ][ org.kde.kstars.ekos.scheduler] - Option to sort jobs based on priority and altitude is false
[2021-03-31T21:49:47.910 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Schedule attempt #1 for 35865-second job 'm1' on row #1 starting at 31/03 21:47, completing at 01/04 07:44."
[2021-03-31T21:49:47.910 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Warning: job 'm1' execution overlaps daylight, it will be interrupted at dawn and rescheduled on next night time."
[2021-03-31T21:49:47.916 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' dark sky score is +0 at 31/03 21:49"
[2021-03-31T21:49:47.917 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' altitude score is +13 at 31/03 21:49"
[2021-03-31T21:49:47.920 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' Moon separation score is +20 at 31/03 21:49"
[2021-03-31T21:49:47.921 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' has a total score of +33 at 31/03 21:49."
[2021-03-31T21:49:47.930 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' on row #1 passed all checks after 1 attempts, will proceed at 31/03 21:47 for approximately 35865 seconds, marking scheduled"
[2021-03-31T21:49:47.930 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Schedule attempt #1 for 27945-second job 'M5' on row #2 starting at 02/04 00:37, completing at 02/04 08:23."
[2021-03-31T21:49:47.934 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'M5' dark sky score is +0 at 31/03 21:49"
[2021-03-31T21:49:47.935 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'M5' altitude score is -1000 at 31/03 21:49"
[2021-03-31T21:49:47.935 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'M5' has a total score of -1000 at 31/03 21:49."
[2021-03-31T21:49:47.941 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'M5' on row #2 passed all checks after 1 attempts, will proceed at 02/04 00:37 for approximately 27945 seconds, marking scheduled"
[2021-03-31T21:49:47.941 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' dark sky score is +0 at 31/03 21:49"
[2021-03-31T21:49:47.943 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' altitude score is +13 at 31/03 21:49"
[2021-03-31T21:49:47.944 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' Moon separation score is +20 at 31/03 21:49"
[2021-03-31T21:49:47.944 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' has a total score of +33 at 31/03 21:49."
[2021-03-31T21:49:47.949 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' is selected for next observation with priority #10 and score 33."
[2021-03-31T21:49:48.848 PDT INFO ][ org.kde.kstars.ekos.scheduler] - Ekos communication status is 0 Starting Ekos...

The first job gets interrupted (properly), but the 2nd job still is scheduled for the next day.
[2021-03-31T23:03:36.714 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' current altitude (25.11 degrees) crossed minimum constraint altitude (0.00 degrees), marking idle."

[2021-03-31T23:03:36.721 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - Job ' "m1" ' is stopping current action... 13
[2021-03-31T23:03:36.723 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' is stopping guiding..."
[2021-03-31T23:03:36.777 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - Find next job...
[2021-03-31T23:03:37.878 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' 15x120\" Blue requires a dither procedure."
[2021-03-31T23:03:37.878 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' 15x120\" Red requires a dither procedure."
[2021-03-31T23:03:37.879 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' 15x120\" Green requires a dither procedure."
[2021-03-31T23:03:37.879 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' 8x240\" Ha requires a dither procedure."
[2021-03-31T23:03:37.879 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' 15x120\" Blue requires a dither procedure."
[2021-03-31T23:03:37.880 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' 15x120\" Red requires a dither procedure."
[2021-03-31T23:03:37.880 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' 15x120\" Green requires a dither procedure."
[2021-03-31T23:03:37.880 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' 8x240\" Ha requires a dither procedure."
[2021-03-31T23:03:37.880 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' 15x120\" Blue requires a dither procedure."
[2021-03-31T23:03:37.881 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' 15x120\" Red requires a dither procedure."
[2021-03-31T23:03:37.881 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' 15x120\" Green requires a dither procedure."
[2021-03-31T23:03:37.881 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' 8x240\" Ha requires a dither procedure."
[2021-03-31T23:03:37.881 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' 15x120\" Red requires a dither procedure."
[2021-03-31T23:03:37.881 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' 15x120\" Blue requires a dither procedure."
[2021-03-31T23:03:37.882 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' 15x120\" Green requires a dither procedure."
[2021-03-31T23:03:37.882 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' 8x240\" Ha requires a dither procedure."
[2021-03-31T23:03:37.883 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' 15x120\" Red requires a dither procedure."
[2021-03-31T23:03:37.883 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' 15x120\" Green requires a dither procedure."
[2021-03-31T23:03:37.883 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' 15x120\" Blue requires a dither procedure."
[2021-03-31T23:03:37.888 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' estimated to take 09h 57m 45s to complete."
[2021-03-31T23:03:37.892 PDT INFO ][ org.kde.kstars.ekos.scheduler] - Option to sort jobs based on priority and altitude is false
[2021-03-31T23:03:37.898 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Schedule attempt #1 for 35865-second job 'm1' on row #1 starting at 31/03 23:03, completing at 01/04 09:01."
[2021-03-31T23:03:37.898 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Warning: job 'm1' execution overlaps daylight, it will be interrupted at dawn and rescheduled on next night time."
[2021-03-31T23:03:38.025 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' is scheduled to start at 01/04 12:50, in compliance with altitude and Moon separation requirements."
[2021-03-31T23:03:38.025 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Schedule attempt #2 for 35865-second job 'm1' on row #1 starting at 01/04 12:50, completing at 01/04 22:48."
[2021-03-31T23:03:38.028 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' is scheduled to start at 01/04 21:00, in compliance with night time requirement."
[2021-03-31T23:03:38.029 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Schedule attempt #3 for 35865-second job 'm1' on row #1 starting at 01/04 21:00, completing at 02/04 06:57."
[2021-03-31T23:03:38.030 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Warning: job 'm1' execution overlaps daylight, it will be interrupted at dawn and rescheduled on next night time."
[2021-03-31T23:03:38.032 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' dark sky score is +0 at 31/03 23:03"
[2021-03-31T23:03:38.034 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' altitude score is -1000 at 31/03 23:03"
[2021-03-31T23:03:38.035 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' has a total score of -1000 at 31/03 23:03."
[2021-03-31T23:03:38.042 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' on row #1 passed all checks after 3 attempts, will proceed at 01/04 21:00 for approximately 35865 seconds, marking scheduled"
[2021-03-31T23:03:38.042 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Schedule attempt #1 for 27945-second job 'M5' on row #2 starting at 02/04 00:37, completing at 02/04 08:23."
[2021-03-31T23:03:38.045 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'M5' is scheduled to start at 02/04 07:02, 300 seconds after 02/04 06:57, in compliance with previous job completion requirement."
[2021-03-31T23:03:38.045 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Schedule attempt #2 for 27945-second job 'M5' on row #2 starting at 02/04 07:02, completing at 02/04 14:48."
[2021-03-31T23:03:38.050 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'M5' is scheduled to start at 02/04 21:00, in compliance with night time requirement."
[2021-03-31T23:03:38.050 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Schedule attempt #3 for 27945-second job 'M5' on row #2 starting at 02/04 21:00, completing at 03/04 04:45."
[2021-03-31T23:03:38.086 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'M5' is scheduled to start at 03/04 00:33, in compliance with altitude and Moon separation requirements."
[2021-03-31T23:03:38.086 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Schedule attempt #4 for 27945-second job 'M5' on row #2 starting at 03/04 00:33, completing at 03/04 08:19."
[2021-03-31T23:03:38.089 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'M5' dark sky score is +0 at 31/03 23:03"
[2021-03-31T23:03:38.089 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'M5' altitude score is -1000 at 31/03 23:03"
[2021-03-31T23:03:38.089 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'M5' has a total score of -1000 at 31/03 23:03."
[2021-03-31T23:03:38.092 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'M5' on row #2 passed all checks after 4 attempts, will proceed at 03/04 00:33 for approximately 27945 seconds, marking scheduled"
[2021-03-31T23:03:38.092 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' dark sky score is +0 at 31/03 23:03"
[2021-03-31T23:03:38.093 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' altitude score is -1000 at 31/03 23:03"
[2021-03-31T23:03:38.093 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' has a total score of -1000 at 31/03 23:03."
[2021-03-31T23:03:38.093 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' is selected for next observation with priority #10 and score -1000."
[2021-03-31T23:03:38.865 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'm1' scheduled for execution at 01/04 21:00. Observatory scheduled for shutdown until next job is ready."

To see if there some other issue, I removed job M1 and just tried to schedule M5. It works fine (scheduling for tonight after rising above the artificial horizon obstruction).

M5 starting at 01/04 00:42, completing at 01/04 08:28."

[2021-03-31T23:10:05.095 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Scheduler started."
[2021-03-31T23:10:05.102 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - Scheduler started.
[2021-03-31T23:10:06.196 PDT INFO ][ org.kde.kstars.ekos.scheduler] - Option to sort jobs based on priority and altitude is false
[2021-03-31T23:10:06.199 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Schedule attempt #1 for 27945-second job 'M5' on row #1 starting at 01/04 00:42, completing at 01/04 08:28."
[2021-03-31T23:10:06.200 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'M5' dark sky score is +0 at 31/03 23:10"
[2021-03-31T23:10:06.201 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'M5' altitude score is -1000 at 31/03 23:10"
[2021-03-31T23:10:06.201 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'M5' has a total score of -1000 at 31/03 23:10."
[2021-03-31T23:10:06.211 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'M5' on row #1 passed all checks after 1 attempts, will proceed at 01/04 00:42 for approximately 27945 seconds, marking scheduled"
[2021-03-31T23:10:06.212 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'M5' dark sky score is +0 at 31/03 23:10"
[2021-03-31T23:10:06.212 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'M5' altitude score is -1000 at 31/03 23:10"
[2021-03-31T23:10:06.212 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Job 'M5' has a total score of -1000 at 31/03 23:10."
[2021-03-31T23:10:06.213 PDT DEBG ][ org.kde.kstars.ekos.scheduler] - "Job 'M5' is selected for next observation with priority #10 and score -1000."
[2021-03-31T23:10:07.213 PDT INFO ][ org.kde.kstars.ekos.scheduler] - "Sleeping until observation job M5 is ready at Thu Apr 1 00:42:41 2021..."

On Wed, Mar 31, 2021 at 10:07 PM Hy Murveit <murveit at gmail.com> wrote:

I worked on this and came up with a solution I like, that I've put in MR 263

https://invent.kde.org/education/kstars/-/merge_requests/263

The scheduler integration was straightforward...basically this is the same as the min-altitude constraints that already work fine. It's just that the scheduler needs to pass the azimuth it's considering as well into a method I created that computes the altitude constraint for the given azimuth.

I modified the Artificial Horizon UI and feature (with Jasem's blessing). It now does not represent polygons at all, but rather a sequence of points defining line segments in azimuth/altitude coordinates (above it in altitude is clear, below is obstructed). If there are multiple lines, use the highest. Entering the points for the line segments is easy...you can enter the points by clicking on the skymap--easy, that is, if you've also created a terrain backgound for your location (my previous MR) so that it's obvious where the obstructions are.

I'd like to add some unit testing, but other than that, seems to work. Testing tonight with my telescope -- gave it some targets with the obstruction issues, and the scheduler seems to be figuring things out.

Check it out, and let me know what you think.
Hy

PS That new feature to append scheduler jobs came in handy. Hadn't used it before. Eric--Thanks for that!

On Thu, Mar 25, 2021 at 1:31 AM Eric Dejouhanet <eric.dejouhanet at gmail.com> wrote:

There are several ways to improve AH:

- import/export mechanisms, instead of asking the end-user to work with sql.
- edit mode :) the safest way to edit AH is currently entering coordinates by hand instead of clicking. This is mostly similar to creating an horizon in the eqmod driver, except the end point must be the same as the beginning point. 
- proper inside/outside rendering of regions, as you noticed. 
- automatic generation of horizon by making a tool count stars on frames taken at grid coordinates. 
- having obstructed regions in the sky instead of horizon obstruction,as you mentioned. 
- obstruction restrictions for the Scheduler, which conflicts with the current design of the plan: observations are currently expected to be able to start at some time of one standard day and to have to finish at some time during that day, after which those tasks either complete or abort. No provision for tasks that may be able to start, stop then restart at some other time before other tasks have even started. This requires a more dynamic scheduling, whose first step is to allow user modification of the plan while Scheduler is running. 

eric.dejouhanet at gmail.com - https://astronomy.dejouha.net

De: murveit at gmail.com
Envoyé: 25 mars 2021 07:23
À: eric.dejouhanet at gmail.com
Répondre à: hy at murveit.com
Cc: hy at murveit.com; kstars-devel at kde.org
Objet: Re: Artificial Horizon

Sorry, don't understand your last line--What is #7?

Also, why would people make scripts when nothing makes use of it? Do users find enough added value just marking up the sky? (I'm new to this and don't really understand it's current value--happy to be educated. I'm really just using my imaging point-of-view).

FWIW, I played with it this evening, and found editing regions buggy. For instance, if you define two regions, and then delete one of them, the one you didn't delete loses its last point and becomes "not closed".  

When defining the polygons, it's necessary to be very exact when clicking the last point (it needs to exactly match the first one). I'd like to be simpler--e.g. the system could easily close the polygon itself. 

Don't think we shoudl constrain the polygons to start (and end) on the horizon--the instructions say one needs to do this. For instance, my telescope's "horizon" never touches the real horizon at any azimuth. 

The UI, though, with the addition of Terrain makes it straight-forward to define a horizon. If the buggy-ness, and the issues with what's inside the polygon and what's outside it were resolved, this could be a reasonable thing to integrate with the scheduler.

Hy

On Wed, Mar 24, 2021 at 11:00 PM Eric Dejouhanet <eric.dejouhanet at gmail.com> wrote:

I don't think any other feature is using AH. It is slightly tedious to create and to edit. Some forum users created database scripts to generate it. #7 is registered in gitlab to make use of it in Scheduler. 

eric.dejouhanet at gmail.com - https://astronomy.dejouha.net

De: murveit at gmail.com
Envoyé: 24 mars 2021 22:50
À: eric.dejouhanet at gmail.com
Répondre à: hy at murveit.com
Cc: kstars-devel at kde.org
Objet: Re: Artificial Horizon

Thanks. I'll take a look at rendering, or perhaps add an "invert" option, if I can't improve the logic.
Does anything outside of the skymap use the artificial horizon?

On Wed, Mar 24, 2021 at 2:39 PM Eric Dejouhanet <eric.dejouhanet at gmail.com> wrote:

1. For various reasons, your horizon may be rendered "from the outside". That's why you don't want to do a single 360 polygon. 

5. Scheduler does not make use of AH. It's been in my pipe for some time, but it's trickier than it seems. The first step would be to have two altitude restrictions, one for setting objects (the current one we have) and one for rising objects. 

eric.dejouhanet at gmail.com - https://astronomy.dejouha.net