[Digikam-users] Off-line Collection Browsing & DB cleanup

[Jean-François Rabasse]

On Sun, 13 May 2012, Nick Anderson wrote:

> Thanks for the response, it would  be great if you could share your scripts.

No problem. I've attached the script I use to this e-mail, in case you 
or others would wish to play with it. It's really simple stuff,
just get it, read it, try it, hack it to suit your personal usage.

> Your workaround seems like it would work ok but generating all the
> thumbnails and keeping them updated seems a pain.

Yes, could be. I should point out that this is just a trick, a hack, 
not a regular way of full working with DK. It only provides a possible
solution to the original question « How can I browse my full collections
even if my original images aren't available at the moment ? »
Nothing more.
And clearly, images files processing/updating is off topic.

Using that kind of trick requires to clearly make a distinction between
images in processing state and images in archived state.
And it's possible to work in a « hybrid » mode, with a DK tree made of
mini images (as placeholders for off-line archived images) and regular
images files, full size, editable, processable.
New subfolders with new images can stay on a local disk several days or
weeks. When processing is considered as done, images go on a removeable
media and are replaced with reduced versions. But reduced versions are
the same images so DK thumbnails and fingerprints remain the same.
(And when, in some occasions, several archived images get edited,
it's always possible to rebuild the reduced versions then ask DK to
rebuild thumbnails and fingerprints on that subset.)

Well actually, I don't think there could be a definitive answer or
workflow. Each individual, each of us, has her/his own way to work and
own usage, and the best way to go is certainly empirical.

> Hopefully the developers see this as a useful feature and stub it in soon. 
> Even read only browsing access would be better than not showing at all. It 
> would be great to have some kind of offline metadata editing with some kind 
> of synchronization feature but I would be content with just read only.

I'd guess it's up to DK developers to answer...
My personal opinion is that it's not a simple work. Digikam is a 
personal images processing software *with* images collections 
management functions, browse, tag, search.
It is not an images database management software, with possible 
local images processing functions, and this makes the difference.
An images bank would require a different model (from a database point
of view), should support temporay off-line data, should also accomodate 
for permament off-line images (e.g. only known via a web URL), and many 
other things.
Moving from one software model to another is a huge task, IMHO.

Also, how many Digikam users get concerned with this off-line issue ?
If we are only a few percent, I doubt this would be worth the task:-)

Regards,
Jean-François
-------------- next part --------------
#!/bin/bash
#
# This script builds an images folders tree, with reduced versions
# of the original images, from a full-sized images tree.
# The aim is to keep on a local disk a whole images collections,
# under a low space format, even when the original images are not
# available on-line.
#
# This script is to be run with two command line arguments :
# - The full pathname of the original images tree.
# - The full pathname for the destination mini tree.
# Both pathnames must start at the system root, "/...." (not checked !)
# The destination cannot be a subdir of the source ! (not checked !)
#
# Hack the buildreduce function to setup mini images building 
# to your personal taste.
# Hack the processfolder function, towards the end, to select which
# kind of images files you wish to process (by extension).
#
# J-F Rabasse, jf at e-artefact.eu
#

# Arguments, base directory, destination directory
# ------------------------------------------------
base="$1"
dest="$2"

if test ! -d "${base}" -o -z "${dest}"
then
	echo 'Usage: <base-directory> <destination-directory>' >&2
	exit 1
fi

# We need the convert program to build reduced images
# ---------------------------------------------------
conv=`which convert 2> /dev/null`
if test ! -x "${conv}"
then
	echo "The convert program is required for this script to work." >&2
	exit 1
fi

# Build reduced version of image
# ------------------------------
# Argument: <subpath-of-image>
#
# NB: arguments to the convert program can be tuned as required.
# Following settings are 300 pixels max bounding box, and 70% quality
# (low quality, high compression).
#
buildreduced()
{
	# Build destination image
	${conv} "${base}/$1" \
		-resize '300x300>' \
		-quality 70	\
		"${dest}/$1"

	# Give new file the same timestamp as original
	touch -r "${base}/$1" "${dest}/$1"

	# We're done
	echo "${dest}/$1 Ok"
}

# Process one images directory
# ----------------------------
# Argument: <subpath-from-base>
#
# NB: subdirectories names are albums names and may contain white spaces.
# We just can't build a shell loop and expect to get files names
# separated with spaces, if names contain spaces.
# The hack is to replace spaces by something special, then replace back
# to the original name.
#
processfolder()
{
	# List directory and protect spaces
	for name in `ls | sed 's/[ ]/@@@@/g'`
	do
		# Rebuild name with possible spaces
		name="`echo ${name} | sed 's/@@@@/ /g'`"

		# Subdirectory, recurse
		if test -d "${name}"
		then

			# Create destination
			mkdir -p "${dest}/$1${name}"

			# Cleanup if wished (get rid of existing mini images)
			# rm "${dest}/$1${name}/*"

			# Sub process source
			cd "${name}"
			processfolder "$1${name}/"
			cd ..

			continue
		fi

		# Regular file, keep only selected images types
		case "${name}" in
		*.JPG|*.jpg|*.png)
			buildreduced "$1${name}"
			;;
		esac
	done
}

# Check/create destination
# ------------------------
mkdir -p "${dest}"
if test ! -d "${dest}"
then
	echo 'Cannot create destination '${dest} >&2
	exit 1
fi

# Process collection tree
# -----------------------
cd "${base}"
processfolder