[KimDaBa] KimDaBa 2.0 is released.

[Robert L Krawitz]

   From: jedd <jedd at progsoc.org>
   Date: Wed, 20 Oct 2004 01:37:13 +1000

   On Tue, 19 Oct 2004 11:01 am, Robert L Krawitz wrote:
    ] I find it particularly valuable with over 5000 images and growing
    ] fairly rapidly (on vacation I typically average about 100 shots per
    ] day, although I had one event this summer where I was the official
    ] photographer when I shot over 600 images in an evening).

    I only averaged 52 shots a day on my last holiday.  But they *were*
    all 3-4mb images .. which should make up for the shortcoming. :)

I usually shoot raw, although for indoor events where contrast won't
be excessive I typically use large fine JPEG.  That night I shot 600 I
used large fine JPEG, and almost filled up 2 GB worth of compact flash.

    ] images from the cameras and convert the raw images to jpeg's for
    ] indexing.  I also mirror the images to a second computer for safety.
    ] Although most of the time I don't actually need it, having the folder
    ] system is very helpful in some cases.

    Do you do all photography in the camera's raw format?  What do you
    do for colour spaces?  My camera gives raw's, tiff's, or jpegs
    .. and the bother with the first two, and the quality of the best
    jpeg setting, convinced me of the merit of taking photos in the
    format that I'm going to ultimately keep them.  I used to use
    PNG's for all my scanned photos, but the lack of EXIF data and the
    comparable quality/space ratio for low-compression jpeg's turned
    me off those, too.

My general attitude is that the raw file is the digital negative,
while the JPEG is a print from that negative.

I do all of my landscape photography in raw.  I did one shoot using
large fine JPEG and regretted it later.  The problem isn't so much
image resolution (JPEG is very good at preserving fine detail; it's
coarse detail it has more trouble with); the problem is the loss of
dynamic range inherent in converting 12-14 bit raw into 8 bit JPEG.  8
bits (which is equal to 8 stops) simply isn't enough dynamic range to
capture good shadow detail without blowing out and/or posterizing
highlights.

I use a heavily hacked-up version of dcraw to do the conversion to
16-bit TIFF, which produces a 38 MB file from my 6.3 MP camera.  This
version of dcraw mitigates the clipping problems that tend to happen
when shooting high dynamic range subject matter.  For example, try
shooting a sunset with a telephoto lens with the sun sinking to the
horizon.  If you look carefully, the result will probably be very
unnatural.  If you underexpose it by a couple of stops, and process
the raw image correctly, you can get natural-looking results, better
than slide film and probably almost as good as color print film.  I'm
happy to send you my version of dcraw if you'd like to try it.

For indoor event shooting, large/fine JPEG is generally adequate.  For
shooting wedding formals it may not be, due to the high contrast
(think tuxedo and gown).  JPEG formats also store more quickly, which
is important when shooting candids or photojournalism, which is my
preferred shooting style for events.

My attitude toward space is that disk space is cheap.  I'm in the
process of scanning old negatives and slides, mostly at 4000 DPI with
a Nikon Coolscan V (LS-50).  The 16-bit TIFF's are about 120 MB each.
Even so, that's about $.10/image.  The 7 MB or so raw files work out
to something like $.006/image.  When I need more disk space, I'll
simply buy it.

    As to folders .. historically I've stored them in sub-directories
    based on the size of CD's, and later DVD's .. to make archiving
    easier.  It's still an effective system since it's inherently
    date-based, and it's a system I adopted some time after I started
    using KimDaBa .. for all my older images (plus those shared by
    friends) I keep them in something like a 'yyyy month dd - verbose
    text description' folder structure.

    I also run a script over all new images to force file time-stamps
    to match the exif time stamp .. useful back when KimDaBa preferred
    to trust file times .. and still occasionally useful now.

My script copies the files (JPEG or raw) over with cp -p, thereby
preserving file modtime.  I then use dcraw and convert to generate a
half or full size JPEG, and then use jhead to set the time and EXIF
data.

I use the same directory scheme the camera uses for simplicity.  I use
keywords to identify specific events.

    ] 1) Export preserves folder layout.  This would be very helpful for
    ]    exporting a set of images for purpose of mirroring them, or for
    ]    backing them up to DVD's in a form that I could easily use to
    ]    restore them later.  The scenario would be that I would select all
    ]    images not previously backed up (by means of a keyword) and export
    ]    them preserving layout, and dump the exported tree to DVD.  This
    ]    would combine very nicely with the export by (hard) link patch that
    ]    I've submitted.

    I think the backup aspect of this would be better left with the
    genuine archival utilities out there.  Certainly exporting, and
    keeping structure, would be a nice feature .. you sure the
    drag-n-drop features don't provide something like this already,
    even indirectly?  (I haven't played with the export functions much
    -- haven't had the need yet.)

Perhaps, although drag & drop doesn't work too well with the command
line scripts I like to use for backup.

    ] 2) Hierarchical folders, that would emulate the actual file storage.

    You mean hierarchical groups .. ?  I'm still trying to get my head
    around what my actual problem is with the existing group system.
    In some cases I want an automatic forcing of group membership for
    some sub-groups, but there's no elegant way to identify these
    things ahead of time.  The extra optional groups are inelegant,
    only insofar as they're not visually seamless in KimDaBa yet, but
    go some way to resolving the problems of 'keyword' being an
    effectively miscellaneous catch-all .. that unavoidably gets
    bloated.  As someone pointed out recently it's easy to get
    duplicates in there .. and quite hard to find them.

Yes, it would require hierarchical groups to implement hierarchical
folders.

    ] 3) A way to associate raw files with jpeg's (and then be able to
    ]    export both).  This would need a bit more knowledge of cameras.

    Do you have both formats right next to each other in the same
    directory structure?  Can I ask why ..?  (As I say, I use jpeg's
    now, but don't know if I'm missing something fundamental about
    going down this path ..  other than saving 14mb of space per
    image.)

Yes, I do.  I use the JPEGS that I generate as index prints, since
kimdaba can't index raw files (and it would be very slow, in any
event).

Again, with JPEG's you're losing a good bit of dynamic range, which is
important in a lot of cases).

    ] 4) Find all images with changed checksums (rather than merely
    ]    recompute the checksum).  Again, this would help identify images
    ]    requiring backup.

    You mean on images that you've modified and then inserted back
    into the system .. ?  Yeah, I think I've done that once, and tried
    to work out an elegant approach for resolving that .. but it's a
    tricky thing that'll likely involve manual work no matter how you
    approach it.  Just because a filename's the same and the
    checksum's different, doesn't mean it's the same image.
    Particularly with digi image names.  So a human is going to have
    to tell KimDaBa what to do in that circumstance anyway.

It shouldn't have to involve manual work.  If the checksum changes,
the image has changed.

-- 
Robert Krawitz                                     <rlk at alum.mit.edu>

Tall Clubs International  --  http://www.tall.org/ or 1-888-IM-TALL-2
Member of the League for Programming Freedom -- mail lpf at uunet.uu.net
Project lead for Gimp Print   --    http://gimp-print.sourceforge.net

"Linux doesn't dictate how I work, I dictate how Linux works."
--Eric Crampton