[Digikam-devel] DcrawParse::getThumbnail

[Will Stokes]

Hello Digikamers,

I recently emailed Gilles directly and he suggested I send our
exchange out to this list in case it was of interest in others. It
follows below...
----------------------------------
I'm the lead developer of Album Shaper, which has
similar and yet very differnt aspirations. :-) More recently I've been
engrosed in developing a spin-off Exif/Iptc viewer/editor called
Reveal. I doubt you've heard of it, I havn't advertised it a lot yet.
In case the following is confusing a screenshot of Reveal might clear
up what it does:
http://albumshaper.sourceforge.net/images/teasers/reveal1.1.jpg

Anyways, Reveal uses the exiv2 library to read/write metadata which
recently has been extended to support Canon's CRW file format. Exiv2
will continue to grow and support other RAW formats, and I'd like to
support viewing/editing metadata for such files in Reveal. In the
summary pane Reveal provides a thumbnail of an image. Reveal 1.0 only
supported JPEG. In order to generate a thumbnail I used IJG code to
quickly scale down the full size image by powers of two, then scaled
to the exact size I wanted using QImage::scaled(). This worked well,
especially since JPEG thumbnails are often not included or not updated
when images are altered. Since QImage does not support loading RAW
files (not suprising) I started looking around for a solution. I
quickly found DCraw and then noticed you have been doing some
experimental work with Digikam and DCRAW for both 8bit and 16bit
support. I simply need a quickly scaled down 8bit image and have
experimenting using your DcrawParse C++ which is defined in
dcraw_parse.cpp. Specifically, I've started generating showing
thumbnails using your DcrawParse ::getThumbnail() function. This all
works just dandy, almost too easily, but I had a few questions for
you:

-It seems your code is simply a masssaged C++ class for the parse code
Dave wrote for dcraw. Dave will continue to update dcraw. I assume you
will continue to patch in such changes into digikam's C++ parse class?
I suppose I could mirror your parse class in Reveal's codebase to pick
up such changes as they are patched in in the future. It's a shame
dcraw is intentially released as a standalone program. This is a
mistake in my opinion. What I would give for a simple libdcraw that
was maintined by Dave himself.

-Your C++ parser throws a ton of warnings when compiled. I suppose
that's technically Dave's fault but annoying none the less.

-I've been able to get "thumbnails" for some NEF files my D70s
produced, as well as for a few CRW files I've found online. That said,
these thumbnails are not exactly thumbnails, rather, they're huge
images (over 2,000 pixels wide, probably the full size image?!). Is
this a bug in the code? I wondered if the code was unable to find a
thumbnail image and instead of was returning a 8bit convereted full
size image but I'm not so sure that is taking place intentialy. Have
you run into this?

-I'm in need of a simple get-resolution method that could tell me the
full size resolution of a given RAW file. Does dcraw have such
functionality in it that does not require actually processing any of
the image data, rather, just reading the headers up top depending on
the RAW file format?

-What exactly are Digikam's RAW plans for the future? RAW seems like a
big pain in the but (much like Exif Makernotes). Manufactueres just
can't seem to agree on how stuff is done, much less release
documentation on their proprietary file formats. DCRAW appears to
simply be aimed at reverse engineering the file format, not applying
stufff like ICC curves. Maybe it is just me but the open source
community (heck, all software developers) needs a single RAW library
that takes care of decoding RAW file formats as well as applying ICC
color curves and programmer instructed white balance and other
options. such things don't really belong in each and every photo like
app that wants to support RAW. How will Digikam graple with this as
time goes on?

A summary  of Gilles response to my comments follow:
-Reveal is not entirely anonymous :-)
-digikam unstable is starting to use exiv2 as well
-digikam developers hope to contribue to exiv2 in the future to help improve it
-Current digikam instable implementation support fully 16 bits images including
all RAW files supported by dcra
-digikam's DCrawParse C++ class needs to be updated using the last
dcraw::parse.c
implementation. This sucks royally because dcraw isn't a pure raw files
management lib... (I concur!)
-digikams' DCrawParse C++ wrapper class isn't a simple c++ port. The
implementation is fully thread safe and any memory leak have been fix
using valgrind.

-As Dave updates dcraw digikam developers will need to continually
patch their DCrawParse class as per his improvements.

-In response to my comment regarding wishing dcraw was simply libdcraw
Gilles wrote "yes, but look on dcraw web site... There are comments
from him about this subject."

> -Your C++ parser throws a ton of warnings when compiled. I suppose
> that's technically Dave's fault but annoying none the less.
-yes. It's not my code (:=)))

->Consider the number of changes that were implemented wouldn't it be
a good idea to fix all the warnings then that DCrawParse generates?
When I compile Reveal now and use this class I got from 0 to 40
warnings which isn't cool. :-( That said, I have very little interest
in forking what digikam developers have effectively already forked
from the dcraw project. Endless forking is not productive. I suppose
making such changes will make it increasingly harder to merge changes
in. instead of making drastic changes like not pasing arguments into
functions that don't need them, there are some simple tricks you can
play to make the compiler happy, for example, instead of defining a
function like this:

void foo(int x,
               int y)
{
stuff that doesn't use x
}

do this

void foo(int, //x not used
               int y)
{
...
}

I havn't investiaged each of the 40 warnings I'm getting, but if some
of them are truely brain dead (like creating variabls and simply not
using them within a function) you cn get aroundthis by simply
resetting the variable to 0 or something...

int x = 6;
x = 0; //CHANGE FROM DCRAW: quiets compiler warnings

I ported some IJG code into Album Shaper and now Clay and had to do
some of this stuff there, although fortunately the IJG code is very
stable (last modified year and years ago at this point), while dcraw
is an ever changing beast.

> -I've been able to get "thumbnails" for some NEF files my D70s
> produced, as well as for a few CRW files I've found online. That said,
> these thumbnails are not exactly thumbnails, rather, they're huge
> images (over 2,000 pixels wide, probably the full size image?!). Is
> this a bug in the code?
-Not sure. Raw file formats is a big "bazard".

> I wondered if the code was unable to find a
> thumbnail image and instead of was returning a 8bit convereted full
> size image but I'm not so sure that is taking place intentialy. Have
> you run into this?
-No. I need to update the code using last parse.c implementation.

> -I'm in need of a simple get-resolution method that could tell me the
> full size resolution of a given RAW file. Does dcraw have such
> functionality in it that does not require actually processing any of
> the image data, rather, just reading the headers up top depending on
> the RAW file format?
-A metadata parser need to be done. I'm currently working on actually.

->For now I've fixed Reveal to use the CRW metadata (translated to
Exif tags on the fly by Exiv2) to fetch the photo dimensions. I don't
like doing this because nothing prevents this data from being
incorrect. Digikam developers might check out the metadata interface
I've cooked up for Reveal, Presenter, and othe Clay based projects.
http://svn.sourceforge.net/viewcvs.cgi/albumshaper/trunk/clay/src/_metadataTools/
http://svn.sourceforge.net/viewcvs.cgi/albumshaper/trunk/clay/src/_metadataTools/metadataInterface.h?view=markup&rev=749

> -What exactly are Digikam's RAW plans for the future? RAW seems like a
> big pain in the but (much like Exif Makernotes).
-RAW files will be supported in read only in image editor. all change must be
saved in TIFF/EP or PNG 16 bits files. DNG file format is planed to the
future

> Manufactueres just
> can't seem to agree on how stuff is done, much less release
> documentation on their proprietary file formats. DCRAW appears to
> simply be aimed at reverse engineering the file format, not applying
> stufff like ICC curves.
-Take a look in last dcraw code. ICC color profile are fully supported.

->I'm real  new to all this ICC stuff. I suppose with my graphics
background and limited knowledge of tone mapping this stuff should be
more obvious to me, but is there a good link someone knows of that
descirbes purpose and application of ICC color curves? I assume the
idea is to convert data that is measured by a linear response sensor
(CCD) to a non-linear scale, either based on your monitor's response
function,  or a model of the perceptual response in the human visual
system, in which case I would think a separate color curve would be
needed for each color channel.

> Maybe it is just me but the open source
> community (heck, all software developers) needs a single RAW library
sure. But i cannot provide any Raw file library. This job must be done by
dcraw author. He is the raw file specialist (:=)))

->This is a real problem in my mind. There is a lot of redundant
working going on out there because a lot of apps want to use dcraw
functionality and are but don't want to include in their disitrbuted
version a separate dcraw binary. Sigh.

> that takes care of decoding RAW file formats as well as applying ICC
> color curves and programmer instructed white balance and other
> options.
> such things don't really belong in each and every photo like
> app that wants to support RAW. How will Digikam graple with this as
> time goes on?
about this point, digiKam instable already support ICC profile workflow. There
are any recent screenshots here :
http://digikam3rdparty.free.fr/Screenshots

-----------------------------------------
I'm happy to hear digikam is embracing exiv2 as well. I've thouroughly
enjoyed using it already. If digikam would like to use my
metadatainterface in the future that is fine as well.

-Will
--------------------------------------------------------
Will Stokes         wstokes (at) gmail.com
Album Shaper   http://albumshaper.sf.net
--------------------------------------------------------