[Digikam-users] Using Proxy Images (WAS: Storing comments and tags outside the image?)

[Dennis Meulensteen]

On Thursday 23 November 2006 18:12, you wrote:
> > Why don't you use proxy images?
> > Keep your digital originals separate, back them up (to a normally
> > unmounted or read-only drive), and run a script to convert them to
> > smaller Jpeg 'proxies'. Then you can use those proxy images for
> > cataloging and browsing. It's a lot faster than cataloging or browsing
> > RAW files on any system. Also 100% safe, unless you botch the backup
> > step.
<SNIP> 
> I'm intrigued by the idea.  I tried to figure out how to configure digikam
> to let me use the "scriptlets", but I don't see any way to do that.  How do
> you get the addition items in the context menu?  I'm running 0.9.0 RC1.  Do
> you have digikam run the backup and conversion when it does a download? 
> Also, would you be willing to post your scripts so I could take a look? 
> Thanks!
<SNIP>

It's actually quite easy, once you know how ;-)

NB: I don't keep my proxy images completely off-line to DigiKam, but store 
them in a sub directory below my proxy images' directory instead. This 
unfortunately means you can't just copy-paste my scripts but will have to 
edit them accordingly if you want greater segregation between the proxies and 
the originals. I depend on self-discipline to never edit my raw images. I 
hardly ever enter the directories containing raw files and don't have 
previews generated for them by DigiKam (KDE).

To put everything into a practical perspective, here is my workflow:

Insert CF card containing images into reader.
Wait for KDE to pop up a window with the contents.
Copy the directory with images into my ~/pictures/2006/ directory
Rename the directory to something appropriate.

Run my backup script. This Rsyncs everything to an off-line disk which is 
mounted by root (prompts for PW) for backups only.

Then I run a script, by simply clicking a shortcut on my desktop, that 
traverses ~/pictures/2006/ looking for NEF (RAW) files that have not been 
moved to their proper ./nef/ subdirs. If it finds any, they are moved to 
a ./nef/ directory, which is made if it did not already exist and a preview 
is made in the original location. 
To keep your proxy images apart, you may want to move them to a different 
directory tree altogether, perhaps even to an alternate file system.

Here's that script:

#!/bin/bash
# recurse through directory tree and process all unprocessed nef files
# v. 2.00 (c) Dennis meulensteen, 2006.
# V.2.01 simple conversion if Jpg present with unprocessed NEF

fnListDir () {
#Recurse through dir and all its subdirs only acting on files
	cd "$1"	
	
	for item in *
	do
		if [ -d "$item" ] 	#check if this item is a directory
		then
			echo "Directory: $item"
			fnListDir "$item" #recurse into the directory
		else 
			#echo "File: $item"
			fnEvaluate "`pwd`/$item"
		fi

	done
	cd ..

}

fnEvaluate () {
# Finds out what to do based on the extension of the file and the dir it is in

	FPATH=${1%/*} # get: /tmp/my.dir (like dirname)
	FILE=${1##*/} # get: filename.tar.gz (like basename)
	BASE=${FILE%%.*} # get: filename 
	EXT=${FILE#*.} # get: tar.gz
	DIRTYPE=${FPATH##*/} # get: the containing directory

	if [ "$EXT" == "nef" ] || [ "$EXT" == "NEF" ]
	then	
		if [ "$DIRTYPE" == "nef" ] || [ "$DIRTYPE" == "NEF" ]
		then
			#echo "Skip processed NEF" 
			if [ ! -f "../$BASE.jpg"  ] && [ ! -f "../$BASE.JPG"  ]
			then
				echo "REFRESHING MISSING jpeg Preview Image"
				fnDoConversion "$FPATH" "$FPATH/../$BASE.jpg" "$1"
			#else
				#Enable to temporarily refresh all JPEGS:
				#rm -f "../$BASE.jpg"
				#fnDoConversion "$FPATH" "$FPATH/../$BASE.jpg" "$1"
			fi

		else
			echo "FOUND UNPROCESSED FILE $FPATH/$FILE"
			fnProcessNef "$FPATH" "$FILE" "$BASE"
		fi
	#else
	#	echo "Found Other file!!"
	fi
}

fnProcessNef () {
# Takes a file and:
# places it in the appropriate subdir, 
# then generates preview image
	cd "$1"

	if [ ! -d "$1"/nef ] # if there is no directory for the file to be moved 
to...
	then
		echo "MAKE NEW NEF DIR: $1/nef"
		mkdir "$1/nef" # make one
	fi
	mv "$1/$2" "$1/nef/$2" #put it in there!

	#skip conversion if jpeg already exists! (shot RAW+JPEG)
	if [  -f "./$3.jpg"  ] || [  -f "./$3.JPG"  ] 
	then
		echo "RESIZE jpeg Preview"
		nice convert -quality 85 -scale 1520x1520 "./$3.jpg" "./$3.jpg"  
	else
		echo "CREATE jpeg Preview"
		fnDoConversion "$1" "$1/$3.jpg" "$1/nef/$2"
	fi
	 
}

fnDoConversion () {
#Params: out path, output file, input file
	nice 
ufraw-batch --shrink=2 --out-type=jpeg --compression=85 --interpolation=bilinear --out-path="$1" --output="$2" --overwrite "$3"

}

##############################################################
# MAIN
# recurse through the dir looking for files to be processed
fnListDir "$1"

exit 0






After this, I start DigiKam and it indexes the new files.

In order to get the context menu to contain my custom scripts, I associate 
them with the .jpg extension.

First you need a scriptlet (I keep these in my ~/bin directory which has exec 
permissions).
Here's one example:

#!/bin/bash

BASEDIR=`dirname "$1"`
BASENAME=`basename "$1"`

NEFPATH="$BASEDIR"/nef
#OUTPATH="$BASEDIR"/output
OUTPATH="/home/dennis/pictures/output"
NEFNAME="$NEFPATH/${BASENAME%jpg}"nef

	if [ ! -d "$OUTPATH" ]
		then mkdir "$OUPATH"
	fi

	ufraw --out-path="$OUTPATH" "$NEFNAME"

This scriptlet is triggered by a right-click from the thumbnail view inside 
DigiKam (or any other KDE app, for that matter.). It gets passed the location 
of the jpeg proxy image, and calculates the location of the original from 
there. In my case it works by simply appending /nef/ to the directory the 
proxy image is in. 

To set up the association between the scriptlet and .jpg files you need to do 
the following:
Save the script. 
Go to Konqueror, [Settings], [Configure Konqueror...] and choose [File 
Associations] from the list. 
Type .jpeg into the search field and click on the entry [-] -image - jpeg. On 
the right you will see a box with Application preference order, click 
[Add...]. Now locate your scriptlet, select it and click [Add]. The system 
configuration will be saved. 
Click [Edit...] 
Under the [Application] tab check that the command listed has the form:
sh /home/dennis/bin/Nef2UFRaw.sh
Give it an appropriate name, like:
Convert Assoc. NEF
hit [OK].

From now on, the option "Convert Assoc. NEF" will be listed in the context 
menu, under [Open With...], for any jpg file you might encounter in any KDE 
app, including DigiKam.

This way you can easily use your proxy images to access the raw files. The 
proxies are so small you probably won't ever need to back them up to DVD. You 
can leave them and let them act as containers for modified metadata and as 
off-line preview images in case you need to move the raw files to DVD. The 
only thing my scripts don't do is update the proxy images whenever the raw 
file is edited. (I never edit the originals anyway).


Dennis

NB: The settings I have in the above scripts currently don't preserve Exif 
data anymore. I haven't gotten around to fixing that. It has to do with the 
parameters I pass to UFRAW, my preferred raw converter. It would probably 
take ten minutes to fix, but I can't be bothered right now.