Exposure Calculator

[Warren]

Hey Joseph,

Again, I think what you’re computing is a *minimum* acceptable exposure
duration, not an *optimum* exposure duration.

I think a few older sources / books / websites have muddied the water on
this by calling it an “optimum.” That’s a holdover from CCDs, where read
noise was a front-and-center concern, which forced people into longer
exposures, which are more difficult to shoot.

These days, no one is forced into long exposures anymore. Hy’s result of 12
seconds is correct — in his environment, an exposure of only 12 seconds is
sufficient for the sky glow to swamp the read noise of his modern sensor.

Some people will find that your calculator says they should shoot 5 second,
or even 2 second subs, which quickly becomes unreasonable due to dataset
size and computational burden.

The situation we’re in today is that we are no longer forced to shoot long
exposures — especially not in urban and suburban environments. Modern
cameras have such low read noise that even very short exposures swap the
read noise.

This is just another way to say that read noise is becoming increasingly
irrelevant, which means that more practical concerns, like dataset size and
computational burden, become dominant.

On Fri, May 19, 2023 at 3:48 PM joseph.mcgee at sbcglobal.net <
joseph.mcgee at sbcglobal.net> wrote:

> Hi Warren,
>
> Thanks for the clarification.
>
> The documentation I will be providing on the calculator does address some
> of these issues with regard to both short exposures, and large stacks and
> the conversely extremely long exposures. Part of this was covered in the
> pdf I sent in response to Hy's email.
>
> The tool does have an input that can be used in cases where the resulting
> exposure time produced is an extreme value. (Hy had discovered this in his
> tests).  The noise increase % effects a bias in the calculation between
> light pollution electrons and read-noise electrons. An adjustment to the
> value will effect the calculated exposure time, and consequently the stack
> size.  The default value of 5% was from one of Dr Glover's presentations.
> (I need to review his presentation to find what influenced his selection of
> this 5% value.)
>
> In response to Hy's email I had suggested that he try lowering gain along
> with a less severe lowering of the noise increase %. This is because I'm
> not certain about whether large changes to the input for the noise increase
> % would be steering the calculation away from a real optimal value.  I
> suspect that they would.
>
> If a user wishes to avoid a large stack of short exposures, and favor
> longer exposures they can lower the noise increase %. The value cannot be
> set to 0 because it is a divisor in part of the calculation. But I
> established the lower limit at 0.05.
>
> The effect of the noise increase value is predictable with regard to the
> exposure time and stack size.  It is a direct inverse relationship to
> sub-exposure time, so halving the noise increase % should double the
> exposure time, and it is therefore a direct relationship to the number of
> exposures in a stack for a given planned session; halving the noise
> increase % should halve the count of exposures in a stack.
>
> But as I said, I'm just not certain whether such a change to this input is
> really appropriate.  It seems to me that this might be defeating the
> purpose of the "optimal" concept of the calculator.
>
> So in the case that Hy provided, the calculator can be biased to produce
> longer exposures. Using an input of 0.25% for noise increase would have
> brought his sub-exposure up to more than 4 minutes, and lowered the stack
> size for 11 planned hours down to just 160 images. The value can be taken
> to an extreme and force the calculation up past a 20 minute sub-exposure by
> setting the noise increase % down to 0.05.
>
> But wouldn't such changes just be saying let the light-pollution
> over-whelm my signal much more than read-noise?  Would we still call this
> result an "optimal" exposure?
>
> So we should probably accept the fact that "optimal" might not be easily
> achievable, and compromises may be necessary.
>
> If large reductions are used to the input for noise increase % to raise
> exposure time, then the result is a compromise, probably due to other
> concerns like storage capacity and post-processing time.
>
> If large increases are needed to the input for noise increase % to lower
> exposure time, then the result is a compromise, probably due to other
> concerns like guiding issues, weather, and satellite or air traffic.
>
>
>
> On 5/19/23 11:44, Warren wrote:
>
> Hey Joseph,
>
> Dr. Glover's presentation was not wrong in any way at all -- it was all
> factual, empirical stuff that we can all agree on.
>
> On the other hand, I think it was incomplete. He ends his presentation by
> saying that there's a point where shorter subs start to significantly hurt
> you, while longer subs provide almost no benefit. Speaking strictly from an
> SNR perspective, he's right, but he fails to mention any of the
> practical downsides of short exposures.
>
> On the one hand, very short exposures are great. You can eliminate small
> passing clouds, moments of atmospheric turbulence, wind gusts that jiggle
> the telescope, etc. in the most surgical way, eliminating only the smallest
> amount of bad data. In the limit, shooting shorter and shorter subs is
> called "lucky imaging," and it's frequently used with planetary observation.
>
> There is a huge downside to taking a billion 1 second exposures, though --
> you'll fill up your storage, and your computer will melt, but you'll
> probably achieve essentially the same result as you would with a more
> modest number of 2 or 5 minute exposures.
>
> There are also workflow considerations. It's much easier to build a dark
> and bias library for a small number of standard exposure durations, instead
> of painstakingly customizing darks and bias frames for every target and sky
> condition.
>
> For typical astrophotographers in urban and suburban environments with
> modern low-noise cameras, it's true that subs longer than 5-10 seconds
> cease to provide any real benefit in terms of SNR, but there are typically
> more urgent practical considerations, e.g. the size of your datasets and
> the time required to process them.
>
> You could say that we are lucky to live in a time where hobbyist cameras
> are so good that 5 second exposures are sufficient to swamp the read noise.
> That doesn't mean we should shoot 5 second exposures -- it means we should
> luxuriate in the flexibility we now have to choose exposure durations of
> almost any length which suits our workflow, our weather conditions, and our
> mount and guiding capabilities.
>
> On Fri, May 19, 2023 at 10:42 AM joseph.mcgee at sbcglobal.net <
> joseph.mcgee at sbcglobal.net> wrote:
>
>> Thanks for the feedback Warren,
>>
>> I'm a little unclear on your concern about the usefulness of the
>> calculator.  But a large part of Dr Glover's presentations seems to be
>> directed to getting astro-photographers to consider using shorter
>> sub-exposures and larger stacks.
>>
>> If you believe this to be incorrect, or less than "optimal", maybe we can
>> work together to come up with an alternate user-selectable calculation
>> model that can be added to this tool.  I would just need this to be
>> described in such a way that I can implement.
>>
>> I'd also be curious to see if folks would run this calculator to compare
>> their experiences.
>>
>> Here's a process that might be helpful to determine the value of the
>> calculator.
>>
>> Pick one of your images, or just a channel used in an image that you
>> consider to be good quality.
>>
>> Set up the calculator with equipment, the conditions and the gain setting
>> that you used for the imaging.
>>
>> Try to adjust the noise increase % so that calculator exposure time is
>> close to the sub-exposure that you used for the image. (It might be tough
>> to get a perfect match, close is good enough).
>>
>> How where did the noise increase % value end up?  Very far from the
>> default 5%?
>>
>> Look at the stack grid to find the closest exposure count to what you
>> used in the stack.  What is the Ratio on that line?
>>
>> ---
>>
>> So here's a long description and details from my learning experience a
>> few years ago that lead to my research into sub-exposure calculations.
>> (Keep in mind that I still consider myself to be very much a novice in this
>> hobby.)
>>
>> As I was first learning in my backyard using a one shot color ASI-071MC,
>> with an f/5.5 refractor. (I typically set the camera gain at 50). I tried
>> imaging at the 3 to 4 minute exposure times that I saw recommended on
>> forums.  The results were awful, and very noisy.  I then purchased both a
>> multi-band filter (Optolong l-Enhance for nebulae) and a light pollution
>> filter (Optolong l-Pro for galaxies). But even after weeks of trial and
>> error, I found that using the l-Pro filter for example, I still had to
>> reduce my exposure times to about 60 seconds with no moon, and about 30
>> seconds with a 1/2 moon.  In these conditions, to get an image that I
>> considered acceptable required about 6 hours for the stack.
>>
>> There's a darker site in the mountains about 90 minutes drive from my
>> home. I only make that trip around a new moon.  My trial and error process
>> there included exposures up to 10 minutes, but even at 5 to 6 minute subs
>> there was excess noise.  I settled on exposures that were 3 to 4 minutes;
>> and I could get a result that was good enough to show to my friends and
>> family, on a stack with just 2 to 3 hours of imaging.
>>
>> This experience triggered the research which lead me to Dr Glover's
>> presentations. I used Dr Glover's equations initially on spreadsheet and
>> later in a Java app. The sub-exposure time from those computations matched
>> my experience fairly closely.
>>
>> I've since measured the SQM in my backyard on a new moon night as 19.3,
>> and about 18.5 with a half moon.  So lets look at what the calculation says
>> for these conditions using the l-Pro filter (I estimate that the l-Pro is
>> passing about 165nm), and I'm leaving the noise increase % at the default
>> 5% recommended by Dr Glover:
>>
>> In my backyard with a new moon, the calculated exposure is 69 seconds;
>> just slightly higher than the 60 seconds I found with trial and error in
>> these conditions.
>>
>> Back then I was still employed with limited available time, so I had been
>> limiting my stacks to what I could get in a single night.  With 6 hours of
>> imaging the calculator shows a ratio (quality) of about 80.  That a ratio
>> of 80 was good enough for me to share with my friends and family.  But in
>> looking at the stacking data I see that the quality is still climbing well;
>> going to a 7th hour would improve the quality by 8%, that might have have
>> been worth doing.
>>
>> But at some point we have to weigh the cost in time vs benefits of longer
>> stacks.  The quality improvement at 20 to 21 hours is not so great; the
>> gain in quality would only be 2.4% for that added hour, and it would be a
>> stack of nearly 1100 images.
>> Then with a half-moon in my backyard: the calculated exposure matches the
>> 30 seconds that I found I needed with trial and error. But to be honest I
>> was never able to get a very good galaxy image around a half-moon from my
>> backyard.  But now it's clear from the calculator that I would need about
>> 14 hours in these conditions to reach a ratio of just 80.
>>
>>
>>
>> Now at the darker site near my home, (I've not yet measured the SQM at
>> this site, but a light pollution map says it is 20.5):
>>
>> The calculation shows a sub-exposure of 221 seconds, that is right in the
>> middle of the 3 to 4 minute range I found with trial and error.  And with
>> just 3 hours stacking the ratio (quality) shows 101.   I was really happy
>> with images from that site with just 3 hours of stacking.
>>
>> So let's run one more calculation for a very dark sky, SQM 21.96 right on
>> the margin of Bortle 1 & 2.
>>
>> I have not yet experienced such a site, so I cannot make any comments
>> about the calculator's result. But it is showing an optimal sub-exposure of
>> about 14 minutes. It also shows that a stack of just 1 hour, (5 exposures),
>> would easily exceed the quality that I find acceptable to share with my
>> friends and family.
>>
>> I also think it's very interesting to see that quality improvement of
>> adding just a second hour in these conditions; a 34% improvement in quality
>> to go from 1 to 2 hours of imaging in these conditions!  But the
>> diminishing improvements of larger stack are still evident; at the 20 to 21
>> hour time-frame the quality improvement is only 2.2%, (but that is at a
>> ratio of over 500, so it mat not be possible to recognize any noise in this
>> image).
>>
>>
>> On 5/18/23 13:28, Warren wrote:
>>
>> I think a fundamental problem with this approach is that it tells you the
>> *minimum* acceptable exposure duration, which is long enough for some other
>> noise source (likely skyglow) to greatly exceed your sensor’s read noise.
>>
>> This is useful information, but mostly when you’re shooting from a Bortle
>> 1-2, where your sensor’s read noise is potentially the limiting noise
>> source — where 60 minute narrowband subs make sense.
>>
>> For folks in urban and suburban environments, with modern low-noise
>> cameras, any realistic exposure duration (e.g. 60-300 seconds) is
>> sufficient for skyglow shot noise to greatly exceed sensor read noise.
>>
>> On Wed, May 17, 2023 at 10:55 PM Hy Murveit <murveit at gmail.com> wrote:
>>
>>> Joseph,
>>>
>>> Thanks so much for getting the exposure calculator up and running in
>>> KStars. Impressive accomplishment!
>>>
>>> I just tried using it, and have some questions/comments I was hoping you
>>> could address.
>>>
>>> Here's a screenshot, with questions below:
>>> [image: Screenshot 2023-05-17 at 10.13.06 PM.png]
>>>
>>>    - I think I filled in the boxes appropriately above, though not
>>>    sure, please let me know. I tried these values: sky quality 19 (about what
>>>    I've measured at my house), f/8 reflector, full bandwidth (300nm), my ZWO
>>>    ASI1600mm camera at gain 75 (I assume it wants the gain I use for the 1600,
>>>    but I tried other values too), 20 total hours of exposure time desired,
>>>    default noise increase of 5%. It seems to be telling me to take 5956 images
>>>    each 12.09 seconds long, which is obviously not a good answer. Am I doing
>>>    something wrong?
>>>    - Not sure what Stack Time, Stack Noise, and Ratio mean. Are shot
>>>    noise and total noise in electrons? (Need tooltips to help)
>>>    - I was able to get it to give me a reasonable exposure time (e.g.
>>>    about a 2-minutes) if I set Noise Increase % to 0.4, but I really didn't
>>>    know what to put in there, and so used the default was 5%. Do you know, is
>>>    5% a good default for the noise increase? Can we give more guidance on what
>>>    noise increase people should start with?
>>>    - The tool needs better tooltips for pretty much each value that
>>>    needs to be entered.  Most  tooltips say "An implementation of Dr Robin
>>>    Glover's exposure calculation." We can give credit elsewhere (e.g. usually
>>>    done in "About KStars"), but the tooltips should be informative. For
>>>    instance, is gain the actual gain values one enters for the camera, or do
>>>    you mean something like quantum efficiency? Assuming it's the value entered
>>>    to the camera's driver, you should say that "Gain value used for your
>>>    camera". Filter Bandwidth should include units (e.g. nm in this case.).
>>>    - Don't need 3 decimal places for Sky Quality (make it one or two
>>>    decimals). Ditto for focal ratio.
>>>    - Is there some documentation on use somewhere? E.g. can a section
>>>    be added to the handbook? Also, please start a forum thread describing this
>>>    new tool and how you recommend users use it.
>>>
>>> Thanks again,
>>> Hy
>>>
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kstars-devel/attachments/20230519/b5ac9ff4/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: C02t5KPn6YbqnqKb.png
Type: image/png
Size: 69767 bytes
Desc: not available
URL: <http://mail.kde.org/pipermail/kstars-devel/attachments/20230519/b5ac9ff4/attachment-0005.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Pjpyvh0ISGXKukA8.png
Type: image/png
Size: 68866 bytes
Desc: not available
URL: <http://mail.kde.org/pipermail/kstars-devel/attachments/20230519/b5ac9ff4/attachment-0006.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: FEkmtmRA05lDU0en.png
Type: image/png
Size: 68910 bytes
Desc: not available
URL: <http://mail.kde.org/pipermail/kstars-devel/attachments/20230519/b5ac9ff4/attachment-0007.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: daKPnBwkHtfgkkrh.png
Type: image/png
Size: 69807 bytes
Desc: not available
URL: <http://mail.kde.org/pipermail/kstars-devel/attachments/20230519/b5ac9ff4/attachment-0008.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot 2023-05-17 at 10.13.06 PM.png
Type: image/png
Size: 378407 bytes
Desc: not available
URL: <http://mail.kde.org/pipermail/kstars-devel/attachments/20230519/b5ac9ff4/attachment-0009.png>