[Digikam-users] Tutorial: Color Management, Camera Profiles, & Working Spaces
"Sveinn í Felli (IMAP)"
sveinki at nett.is
Sun Jun 1 19:03:03 BST 2008
Great work Elle, useful to many people. Thanks a lot.
Sveinn í Felli
elle stone wrote:
> This tutorial on color management, camera profiles, and working spaces has
> been put together in the hope that it will be commented on, corrected, and
> possibly even incorporated into the digikam handbook. The existing
> information in the handbook is wrong on quite a few counts regarding color
> management, and therefore confusing to a new user. Rather than complain
> about the problems, I thought I would try to offer part of a solution.
> "Color Management" is a complicated subject. Fortunately, as digital
> photographers we only need to know enough to make a few good choices along
> the image editing "pipeline" from camera to final output. Hopefully our
> color-managed image editor and all the other software we use in our "digital
> darkroom" will then do all the "heavy lifting" behind the scenes.
> In this tutorial I will attempt to describe as simply and accurately as
> possible two important color management choices:
> (1)What "camera profile" should I use during the raw conversion process?
> (2)What "working space" should I use while I am editing my image?
> My goal in this tutorial is to put these two very important color management
> choices in the overall context of an understanding of what color management
> actually does. Two other color management choices mentioned only in passing
> (but stay tuned for the next tutorial!) are what monitor/display profile
> should I use? and what do I do about choosing an "output" profile if I want
> to print or email my image or perhaps publish it to the web?
> Every camera is different:
> Digital cameras have an array of millions of little light sensors inside,
> making up either a CCD or a CMOS chip (the difference between "CCD" and
> "CMOS" is very interesting but beyond the scope of this tutorial; if you are
> curious, google is your friend). These light-sensing "pixels" are
> color-blind. So to allow pixels to record color information, each pixel is
> capped by a transparent red, green, or blue lens, usually alternating in
> what is called a "Bayer" array. The whole point of "interpolation" using
> "demosaicing algorithms" such as dcraw's default "AHD" is to "guess" what
> color light actually fell on any given pixel by "interpolating" information
> gathered from that single pixel plus its "neighboring" pixels (see
> http://en.wikipedia.org/wiki/Demosaicing). After interpolation, the raw
> converter software outputs a file (usually a 16-bit tiff) containing a trio
> of interpolated R,G,B values for each pixel in the image.
> The good news regarding today's digital cameras is that the sensors, all
> those little "pixels" on the ccd or cmos chip inside the camera, are capable
> of capturing virtually ALL the visible spectrum. The bad news is that the
> this trio of R,G,B numbers for each pixel in an image, as produced by the
> raw converter from the raw image that the camera stores on the camera card,
> is essentially meaningless until "interpreted" by a camera profile that is
> specific to the particular (make and model of) camera. Why? Because "pixel
> response to light" is the result of lots of camera-specific factors
> including: the nature of the sensor array itself, the precise
> coloring/transmissive qualities of the lens caps, and the particular
> "analog-to-digital conversion" and post-conversion processing that happens
> inside the camera to produce the raw image that gets stored on the card.
> (As an aside, the "analog-to-digital conversion" inside the camera is
> necessary because the light-sensing "pixels" are analog in nature - they
> collect a charge proportionate to the amount of light that reaches them; the
> accumulated charge on each pixel needs to be turned into a discrete, digital
> quantity by the camera's "analog to digital converter").
> The "Universal Translator": your camera profile, the Profile Connection
> Space, and lcms
> So the question for each RGB trio of values in the (let us assume) 16-bit
> tiff produced by (let us assume) dcraw becomes, "What does a particular trio
> of RGB values for the pixels making up images produced by this particular
> (make and model) camera really mean in terms of some "absolute standard"
> referencing some "ideal observer"?" This "absolute standard" referencing an
> "ideal observer" is more commonly called a "Profile Connection Space". A
> "camera profile" is needed to accurately "characterize" or "describe" the
> response of a given camera's pixels to light entering that camera, so that
> the RGB values in the output file produced by the raw converter can be
> "translated" first into an absolute Profile Connection Space (PCS) and then
> from the PCS to your chosen working space.
> As a very important aside, for most of the open source world (including
> digikam), the software used to "translate" from the camera profile to the
> PCS and from the PCS to your chosen "working space" and eventually to your
> chosen "output space" (for printing or perhaps monitor display) is based on
> lcms (the "little color management engine" - see http://littlecms.com). For
> what it's worth, my own testing has shown that lcms does more accurate
> conversions than Adobe's proprietary color conversion engine. Further, for
> almost all raw conversion programs, including commercial closed source
> software such as Adobe Photoshop, the raw conversion is typically based on
> "decoding" of the proprietary raw file done by dcraw. David Coffin, author
> of dcraw, is the hero of raw conversion - without him we'd all be stuck
> using the usually "windows/mac only" proprietary software that comes with
> our digital cameras. For what it's worth, my own testing has shown that
> dcraw's interpolation algorithms (not to be confused with the aforementioned
> "decoding" of the proprietary raw file), if properly used, produce results
> equal or superior to commercial, closed source software. We in the world of
> linux and open source software are NOT second-class citizens when it comes
> to digital imaging. Far from.
> There are two commonly used Profile Connection Spaces - CIELAB and CIEXYZ
> (see http://en.wikipedia.org/wiki/Color_management, section on "Color
> translation", then look up CIELAB and CIEXYZ on wikipedia). Lcms uses the
> camera profile to "translate" the RGB values from the interpolated raw file,
> that is, the tiff produced by dcraw, into the appropriate Profile Connection
> Space (usually CIEXYZ - why "CIEXYZ"? I haven't taken the time to learn).
> A "profile connection space" is not itself a "working space". Rather a PCS
> is an absolute reference space used only for translating from one color
> space to another - think of a PCS as a "Universal Translator" for all the
> color profiles that an image might encounter in the course of its "journey"
> from camera raw file to final output:
> (1)Lcms uses the "camera profile", also called an "input" profile, to
> "translate" the interpolated dcraw-produced R,G,B numbers, which only have
> "meaning" relative to your (make and model of) camera, to a second set of
> R,G,B numbers that only have meaning in the Profile Connection Space.
> (2)Lcms "translates" the Profile Connection Space R,G,B numbers to the
> corresponding numbers in your chosen working space so you can edit your
> image. And again, these "working space" numbers ONLY have meaning relative
> to a given working space. The same "red", visually speaking, is represented
> by different "trios" of RGB numbers in different working spaces; and if you
> "assign" the wrong profile the image will look wrong, slightly wrong or very
> wrong depending on the differences between the two profiles.
> (3)While you are editing your image in your chosen working space using your
> favorite image editing program (which hopefully is digikam!), then lcms
> should translate all the working space RGB numbers back to the PCS, and then
> over to the correct RGB numbers that enable your monitor (your "display
> device") to give you the most accurate possible "display" representation of
> your image as it is being edited. This "translation for display" is done
> "on the fly" and you should never even notice it happening, unless it
> doesn't happen correctly - then the displayed image will look wrong, perhaps
> a little wrong, perhaps really, really, really wrong. Stay tuned for the
> next tutorial (assuming anyone finds this tutorial useful) for details on
> color-managing your monitor display.
> (4)When you are satisfied that your edited image is ready to share with the
> world, lcms "translates" the "working space" RGB numbers back into the PCS
> space and out again to a printer color space using a printer profile
> characterizing YOUR printer/paper combination (if you plan on printing the
> image) or to sRGB (if you plan on displaying the image on the web or
> emailing it to friends or perhaps creating a slide-show to play on monitors
> other than your own.
> To back up a little bit and look at the first color profile an image
> encounters, that is, the camera profile (see (1) immediately above) - dcraw
> can in fact apply your camera profile for you (dcraw uses lcms internally).
> But (i)the generating of the tiff composed of the interpolated RGB values
> derived from the camera raw file, and (ii)the application of the camera
> profile to the interpolated file, are two very distinct and totally
> separable (separable in theory and practice for dcraw; in theory only for
> most raw converters) steps. The dcraw command line output options "-o 0
> [Raw color (unique to each camera)] -4 [16-bit linear] -T [tiff]" tell dcraw
> to output the RGB numbers from the raw interpolation into a tiff WITHOUT
> applying a camera ("input") profile (the words in brackets explain the
> options but should not be entered at the command line). Then, if you truly
> enjoy working from the command line, you can use the lcms utility "tifficc"
> to apply your camera profile yourself. The advantage of doing so is that
> you can tell lcms to use "high" quality conversion (dcraw seems to use the
> lcms default "medium"). The disadvantage, of course, is that applying your
> camera profile from the command line adds one extra step to your raw work
> Where to find camera profiles:
> So where do we get these elusive and oh-so-necessary camera-specific
> profiles that we need to "translate" our interpolated raw files to a working
> color space? The UFRAW website section on color management
> (http://ufraw.sourceforge.net/Colors.html) has a bit of information on where
> to find ready-made camera profiles. It's an unfortunate fact of digital
> imaging that the camera profiles supplied by Canon, Nikon, and the like
> don't work as well with raw converters other than each camera manufacturer's
> own proprietary raw converter. Which is why Bibble and Phase One (and Adobe,
> but ACR "hides" the Adobe-made profiles inside the program code), for
> example, have to make their own profiles for all the cameras that they
> support - keep this "proprietary propensity" of your camera manufacturer in
> mind next time you buy a digital camera.
> But back to finding a camera profile for YOUR camera - the "real" answer
> (assuming you don't find a ready-made profile that makes you happy) is to
> make your own camera profile or have one made for you. There are quite a
> few commercial services who provide profiling services (for a fee, of
> course). Or you can use "LPRof" or "Argyll" to profile your camera
> yourself. I haven't yet walked down that road so I can't speak about how
> easy or difficult the process of profiling a camera might be. But I would
> imagine, knowing how very meticulous the people behind Argyll, LPRof, and
> lcms are about color management, that making your own camera profile is very
> "do-able" and very likely the results will be better than any proprietary
> profile. After all, Canon (and also Bibble and Phase One for that matter)
> didn't profile MY camera - they just profiled a camera LIKE mine.
> Working Spaces:
> So now your raw file has been interpolated by dcraw and you've obtained a
> camera profile and used lcms "tifficc" to "apply" your camera profile to the
> tiff produced by dcraw (or you've asked dcraw to apply it for you). What
> does all this mean? The "real" answer involves a lot of math and color
> science that goes way over my head and likely yours. The short, practical
> answer is that neither the camera profile space nor the Profile Connection
> Space is an appropriate space for image editing. Your next step is to
> choose a "working space" for image editing. And then you (or rather the
> lcms "color management engine" that your open source digital imaging
> software uses) actually perform a "double translation". First lcms uses the
> camera profile to translate the RGB values of each pixel in the
> "dcraw-output-image-without-camera-profile-applied" into the aforementioned
> "Profile Connection Space". Then it translates the RGB values of each pixel
> from the PCS to your chosen working space.
> Confusions and confusing terminology:
> Before talking more about "working spaces", some confusions and confusing
> terminology needs to be cleared up:
> First, sRGB is both a "working" color space and an "output" color space for
> images intended for the web and for monitor display (if you have a spiffy
> new monitor with a gamut larger than the gamut covered by sRGB, obviously
> you might want to reconsider what output profile to use to best take
> advantage of your wonderful and hopefully calibrated and profiled monitor,
> but please convert your image to sRGB before sending it on to your
> friends!). sRGB is also the color space that a lot of home and
> mass-production commercial printers "expect" image files to be in when sent
> to the printer. It is also the color space that most programs "assume" if
> an image does not have an embedded color profile telling the program what
> color space should be used to interpret ("translate") the RGB numbers. So
> if you choose to not use color-management, your color-management "choices"
> are simple - set everything to sRGB.
> Second, all jpegs (or tiffs, if you have an older Minolta Dimage camera)
> coming straight out of a camera (even if produced by point-and-shoots
> cameras that don't allow you to save a raw file) start life inside the
> camera as a raw file produced by the camera's A to D converter. The
> processor inside the camera interpolates the raw file, assigns a camera
> profile, translates the resulting RGB numbers to a working space (usually
> sRGB but sometimes you can choose AdobeRGB, depending on the camera), does
> the jpeg compression, and stores the jpeg file on your camera card. So
> jpegs (or tiffs) from your camera NEVER need to be assigned a camera or
> "input" profile which is then "translated" to a working space via a PCS.
> Jpegs from a camera are already in a working space.
> Third, in case anyone is unsure on this point, note that an "interpolated"
> raw file is no longer a raw file - it has been interpolated and then
> "output" as a tiff whose RGB values need to be "translated" to a working
> space, using the camera profile, the PCS, and lcms.
> Fourth (strictly for future reference), to introduce a bit of commonly
> heard color-management terminology here - the camera profile and your
> printer's color profile are both "device dependent," whereas the working
> space will be "device-independent" - it can be used with any image, with any
> properly color-managed software, without regard for where the image
> Fifth, above I have used the words "translate" and "translation" as a
> descriptive metaphor for what lcms does when it "translates" RGB values from
> one color space to another via the PCS. The usual and correct terminology
> is "convert" and "conversion", which I will use below. The four "methods of
> conversion" from one color space to another are: "perceptual", "relative
> colorimetric", "absolute colorimetric", and "saturation". Which method of
> conversion you should use for any given image processing step from raw file
> to final output image is beyond the scope of this tutorial. The standard
> advice is: when in doubt, use "perceptual."
> Sixth (and again, strictly for future reference),"assign a profile" means
> "change the meaning of the RGB numbers in an image by embedding a new
> profile without changing the actual RGB numbers associated with each pixel
> in the image"; "convert" means "embed a new profile, but also change the RGB
> numbers at the same time so that the "meaning" of the RGB values - that is,
> the "real-world visible color" represented by the trio of RGB numbers
> associated with each pixel in an image - remains the same before and after
> the conversion from one space to another". You should be able to do
> multiple conversions of an image from one working space to another, and with
> a properly color-managed image editor, even though all the RGB numbers in
> the image will change with each conversion, the image on your screen should
> look the same (leaving aside the usually unnoticeable small but inevitable
> changes from accumulated gamut mismatches and mathematical rounding errors).
> However, every time you "assign" a new working space profile rather than
> "convert to" a new working space, the appearance of the image should more or
> less drastically change (usually for the worse).
> Finally, (and this is a crucially important point), color management is NOT
> "only relevant if you shoot raw". Color management affects every stage of
> the image processing pipeline, whether you start with a raw file that you,
> yourself "interpolate and translate" into a tiff, or if you start with a
> jpeg or tiff produced by your camera.
> Copyrighted and "copyleft" working spaces:
> I will take it as given that ALL the ordinarily encountered working spaces,
> such as:
> (1)the several variants of sRGB (see http://www.color.org/v4spec.xalter)
> (2)"BruceRGB" (http://www.brucelindbloom.com)
> (3)the various "ECI" (European color initiative) working space profiles (see
> (4)AdobeRGB, Adobe WideGamutRGB, and Kodak/Adobe ProPhotoRGB (Kodak and
> Adobe ProPhoto are the same, just "branded" differently) and their
> non-branded, non-copyrighted counterparts (Oyranos includes a non-branded
> version of AdobeRGB; see
> (5) and quite a few others that could be added to this list
> are all more or less "suitable" as working spaces. Which working space you
> "should" use depends only and solely on YOU, on YOUR requirements as the
> editor of YOUR digital images with YOUR eventual output intentions (web,
> fine art print, etc).
> However, as a critical aside, if you are using Adobe (or other copyrighted)
> working space profiles, these profiles contain copyright information that
> shows up in your "image exif" information. Lately I've been perusing the
> openicc mailing lists. Apparently lcms can be used to produce nonbranded,
> "copyleft" working space profiles that are just the same as - actually
> indistinguishable from - the branded, copyrighted working space profiles.
> It would be a wonderful addition to digikam if a set of "copyleft" working
> space profiles, including nonbranded, relabelled versions of ProPhotoRGB,
> AdobeRGB, and Adobe WidegamutRGB (perhaps in two "flavors" each: linear
> gamma and the usual gamma), could be bundled as part of the digikam package.
> Which working space: gamma
> Now, the next question is "which working space should I use?" Wikipedia
> says "Working spaces, such as sRGB or Adobe RGB, are color spaces that
> facilitate good results while editing. For instance, pixels with equal
> values of R,G,B should appear neutral. Using a large (gamut) working space
> will lead to posterization, while using a small working space will lead to
> clipping. This trade-off is a consideration for the critical image
> editor" (http://en.wikipedia.org/wiki/Color_management#Working_spaces).
> Well, that quote from wikipedia is about as clear as mud and I don't know
> if I will be able to explain it more clearly, but I will try. "[P]ixels
> with equal values of R,G,B should appear neutral" just means that for any
> given pixel in an image that has been converted to a suitable working space,
> if R=G=B you should see grey or black or white on your screen.
> I am not aware of a list of other technical requirements for a "suitable"
> working space, though undoubtedly someone has produced such a list. But
> most working space profiles are characterized by (1)"RGB primaries" which
> dictate the range of colors, that is, the "gamut" covered by a given
> profile; (2)"white point", usually D50 or D65, which dictates the total
> dynamic range of the working space, from 0,0,0 (total black) to the
> brightest possible white; and (3)"gamma."
> The practical consequences that result from using different "RGB
> primaries", leading to larger or smaller working spaces, are discussed
> below. The practical consequences for different choices for the working
> space "white point" are beyond the scope of this tutorial. Here I will talk
> a little bit about the practical consequences of the working space "gamma"
> (for an excellent article and references, look up "gamma" on wikipedia).
> The "gamma" of a color profile dictates what "power transform" needs to
> take place to properly convert from an image's embedded color profile
> (perhaps your working color space) to another color profile with a different
> gamma, such as (i)the "display" profile used to display the image on the
> screen or (ii)perhaps to a new working space, or (iii)perhaps from your
> working space to your printer's color space. (As an aside, mathematically
> speaking, for a "power transform" you "normalize" the RGB numbers and raise
> the resulting numbers to an appropriate power depending on the respective
> gammas of the starting and ending color space, then renormalize the results
> to a new set of RGB numbers. Lcms does this for you when you ask lcms to
> convert from one color space to another; however, if ALL you are doing is a
> power transform, use imagemagick instead of lcms and just manipulate the RGB
> numbers directly - the results will be more accurate. Now aren't you glad
> that I've kept the mathematics of color management out of this tutorial?)
> One practical consequence of the "gamma" of a working space is that the
> higher the gamma, the more "tones" are available for editing in the shadows,
> with consequently fewer tones available in the highlights. So
> theoretically, if you are working on a very dark-toned ("low key") image you
> might want a working space with a higher gamma. And if you are working on a
> "high key" image, say a picture taken in full noon sunlight of a wedding
> dress with snow as a backdrop, you might want to choose a working space with
> a lower gamma, so you have more available tonal gradations in the
> highlights. But in the real world of real image editing, almost everyone
> uses working spaces with either gamma 1.8 or 2.2.
> As an aside, recently I've heard that some people are trying to
> "standardize" on gamma 2.0. As a very important aside, sRGB and "LStar-RGB"
> are not "gamma-based" working spaces. Rather, sRGB uses a "hybrid" gamma -
> see "http://en.wikipedia.org/wiki/SRGB" for details. And "LStar-RGB" uses a
> luminosity-based "tonal response curve" instead of a gamma value - see
> "http://www.colormanagement.org/en/workingspaces.html" for more information,
> and then google around for more in-depth information.
> In addition to "gamma 1.8" and "gamma 2.2" the only other "gamma" for a
> working space that gets much mention or use is "gamma 1", also called
> "linear gamma". "Linear gamma" is used in HDR (high dynamic range) imaging
> and also if one wants to avoid introducing "gamma-induced errors" into one's
> "regular" low dynamic range editing. "Gamma-induced errors" is a topic
> outside the scope of this tutorial, but see "Gamma errors in picture
> scaling, http://www.4p8.com/eric.brasseur/gamma.html";
> "http://www.21stcenturyshoebox.com/essays/color_reproduction.html" for
> gamma-induced color shifts; and of course Timo Autiokar's somewhat infamous
> website, http://www.aim-dtp.net/.
> Unfortunately and despite their undeniable mathematical advantages, linear
> gamma working spaces have so few "tones" in the shadows that (in my opinion)
> they are impossible to use for editing if one is working in 8-bits, and
> still problematic at 16-bits (though I do use linear gamma working spaces
> myself for some parts of my image editing workflow). When the day comes
> when we are all doing our editing on 32-bit files produced by our HDR
> cameras on our personal supercomputers, I predict that we will all be using
> working spaces with gamma 1; Adobe Lightroom is already using a linear gamma
> working space "under the hood" and Lightzone has always used a linear gamma
> working space.
> Which working space: "large gamut" or "small gamut"
> One MAJOR consideration in choosing a working space is that some working
> spaces are "bigger" than others, meaning they cover more of the visible
> spectrum (and perhaps even include some "imaginary" colors - mathematical
> constructs that don't really exist). These bigger spaces offer the
> advantage of allowing you to keep all the colors captured by your camera and
> preserved by the lcms conversion from your camera profile to the really big
> "profile connection space".
> But "keeping all the possible colors" comes at a price. It seems that any
> given digital image (pictures of daffodils with saturated yellows being one
> common exception) likely only contains a small subset of all the possible
> visible colors that your camera is capable of capturing. This small subset
> is easily contained in one of the smaller working spaces. Using a very
> large working space mean that editing your image (applying curves,
> saturation, etc) can easily produce colors that your eventual output device
> (printer, monitor) simply can't display. So the "conversion" from your
> "working space" to your "output device space" (say your printer) will have
> to "remap" the "out of gamut" colors in your edited image, some of which
> might even be totally imaginary, to your printer color space with its much
> smaller gamut, leading to inaccurate colors at best and at worst to
> "banding" ('posterization' - gaps in what should be a smooth color
> transition, say, across an expanse of blue sky) and "clipping" (e.g your
> carefully crafted muted transitions across delicate shades of red, for
> example, might get "remapped" to a solid block of dull red after conversion
> to your printer's color space).
> In other words, large gamut working spaces, improperly handled, can lead to
> lost information on output. Small gamut working spaces can clip information
> on input. Like Wikipedia says, it's a trade-off. I can offer some
> oft-repeated advice:
> (1)For images intended for the web, use (one of the) sRGB (variants - there
> are several).
> (2)For the most accuracy in your image editing (that is, making the most of
> your "bits" with the least risk of banding or clipping when you convert your
> image from your working space to an output space), use the smallest working
> space that includes all the colors in the scene that you photographed, plus
> a little extra room for those new colors you intentionally produce as you
> (3)If you are working in 8-bits rather than 16-bits, choose a smaller space
> rather than a larger space.
> (4)For archival purposes, convert your raw file to a 16-bit tiff with a
> large gamut working space to avoid loosing color information. Then convert
> this "archival" tiff to your working space of choice (saving the converted
> "working" tiff under a new name, of course). See
> "http://www.21stcenturyshoebox.com/essays/scenereferredworkflow.html" for
> more details.
> The "whys" of these bits of advice regarding "which working space" are
> beyond the scope of this tutorial. See Bruce Lindbloom's excellent website
> (http://www.brucelindbloom.com/, Info, Information about RGB Working Spaces)
> for a visual comparison of the "gamut" (array of included colors) of the
> various working color spaces. See
> "http://www.luminous-landscape.com/tutorials/prophoto-rgb.shtml" and
> "http://www.cambridgeincolour.com/tutorials/sRGB-AdobeRGB1998.htm" for a
> "pro" and "con" presentation, respectively, of the merits of using "large
> gamut" working spaces. And while you are on the "cambrideincolour.com"
> website, check out the tutorial on color management.
> And this concludes my tutorial on color management, camera profiles, and
> working spaces. Once again, please feel free to comment, correct,
> incorporate into the digikam handbook, or ignore altogether. As I already
> said, I couldn't help but notice that the existing information in the
> digikam handbook is wrong on quite a few counts regarding color management
> (regarding which I will post separately). Rather than just complain about
> the problems, I thought I would try my hand at spelling out some theoretical
> background and practical consequences of color management choices regarding
> "camera profiles" and "working spaces".
More information about the Digikam-users