Digikam raw files and darktable

[Juan Jose Casafranca]

I think you are trying to solve the problem in a very complex way. 

First of all, where does digikam currently saves the thumbnails? If it's in 
the database, then, the only thing needed is a way to generate the processed 
raw as a thumbnail and a flag which specify if the thumbnail is up to date. 

This flag es very easy to maintain. When loading the database, the thumbnails 
are not up to date. So darktable (or any other raw processor) will be called 
to generate the thumbnails. When a raw file is opened for edition through the 
digikam interface, then also mark the thumbnail is not up to date. If the user 
opens the file for edition from other place different from the digikam 
interface (opening the raw processor directly for example), then do anything. 
I know that thumbnail stored and the processed image will not match in this 
case, this should be documented. Enable an option to update manually the 
thumbnails for raw files. 

Digikam should have an option to choose which raw processor want the user to 
user. Default one, darktable, raw therapee, whatever. When a user try to edits 
a raw file, then the raw processor selected is opened. 

I dont think that we need to store any extra information in the xmp file. 
Simply, made different software dont override other software xmp tags (as I 
said in other post, I think this is common sense).

Digikam would not treat darktable in a special way. I'm just asking for an 
interface to connect digikam with any raw processor. This interface should 
only need to provide a way to open the raw processor with the selected image 
and a way to get back the image from the raw processor. 

I dont really understand why digikam and raw processors should communicate 
through the xmp file.

So, to summarize:
User selects darktable for raw editing
User opens a raw file in digikam -> darktable is opened
User edits in darktable and close it -> thumbnail is marked as not up to date
Thumbnail is updated calling darktable which returns an image (maybe with a 
LUA script?)
The thumbnail is stored in digikam the same way other jpeg or current raw 
thumbnails are stored

On sábado, 7 de enero de 2017 12:33:43 (CET) Remco Viëtor wrote:
> On samedi 7 janvier 2017 10:31:27 CET Gilles Caulier wrote:
> > I follow this thread, and that i can said is : RAW workflow is terrific
> > and
> > very complex. I spare few years to undstand all main subtilities to
> > implement a first suitbale support of RAW files in digiKam.
> > 
> > Read the contextual responses below for the next...
> > 
> > 2017-01-07 10:15 GMT+01:00 Remco Viëtor <remco.vietor at wanadoo.fr>:
> > > On samedi 7 janvier 2017 03:42:35 CET Juan Jose Casafranca wrote:
> > > > I know that darktable writes not only tags, rating etc, but also the
> > > > process history.
> > > > 
> > > > That's why I want digikan to call darktable when it founds a raw file.
> > > > (...)
> > > 
> > > Wouldn't it be easier to get an option in Darktable to write a thumbnail
> > > file
> > > to disk when leaving darkroom mode or changing file there? And then add
> > > a
> > > link
> > > to that thumbnail in the XMP file. Basically, you add an API for
> > > external
> > > programs to provide a thumbnail image (and afaik, KDE has a standardised
> > > way
> > > to store thumbnails in a directory).
> > 
> > This API is limited even if it's standardized by Open Desktop. digiKam
> > drop
> > this support since a while for multiple technical reasons, as using
> > Wavelets compression to reduce space instead PNG, using a dedicted
> > database
> > to handle disconnected removable media, be able to store thumb in a remote
> > database, support more than 256x256 thumbnails size, etc...
> > 
> > So no KDE API here. In all case, we want to reduce to the max KDE
> > dependencies for the future to be porta le everywhere and reduce
> > complexity.
> > 
> > Note : KDE thumbnials use KIO : we drop this support since 5.0, which DO
> > NOT WORK outside Linux, and cannot be bundled in AppImage. No KIO in
> > digiKam for the future...
> 
> OK, so that is not a viable option. But a clear way for external programs to
> get information (like thumbnails) back to Digikam would be helpful, and the
> sidecar could be a channel for that.
> 
> > > And Darktable already generates preview
> > > images/thumbnails (top left in the darkroom window, and of course in
> > > lighttable mode). Better to reuse those if at all possible.
> > 
> > Where are stored this preview image ? In a dedicated place... This will be
> > another puzzle to interface digiKam with that...
> 
> Ideally, Digikam shouldn't have to worry about if and where those thumbnails
> are stored internally by Darktable. But the code to *generate* those
> thumbnails is in place already, what's missing is a way to *export* those
> thumbnails for use with external programs (i.e. Digikam).
> 
> > > It's clear that Digikam's editor will not be able to generate a
> > > thumbnail
> > > from
> > > the original RAW
> > 
> > Wrong. digiKam use libraw and extract JPEG preview from RAW file to
> > generate thumbnails...
> > 
> > > and a Darktable XMP. But starting Darktable for each and
> > > (...)
> 
> Sorry, you cut my phrase at the wrong point...
> What I wanted to say is that Digikam cannot repeat the processing Darktable
> does, i.e. cannot generate a thumbnail by combining the information from the
> RAW file and the XMP file (of course Digikam can generate a thumbnail from
> the RAW file only, it's doing just that all the time...)
> 
> > A possible solution is to use IPTC field stored in XMP. There is a IPTC
> > preview tag to host JPEG preview. It's limited to 256Kb of data which is
> > enough to do the job. Problem : Old IPTC support is, in XMP i don't know,
> > especially to host binary data in XML.
> > 
> > Other problem is to explode the sidecar XMP file size. This is why we use
> > a
> > dedicted database....
> > 
> > > Another problem with that would be all the other RAW processors out
> > > there.
> > > Why
> > > would Digikam treat Darktable in a special way and ignore all the
> > > others?
> > 
> > Remember that we use already a RAW processor : libraw. there are plenty of
> > new feature in this processor not yet used in digiKam. The best way is to
> > implment new RAW feature in digiKam, at least to import RAW data in
> > editor.
> 
> Possible, but not relevant to this subject (IMO): there are many reasons why
> someone would use Digikam for DAM, and another program for RAW development.
> 
> The point here is to get a *representation* of the image, as produced by
> that other program, in the Digikam thumbnails view.
> 
> So, basically, what I'd aim for is:
> - define a way to get thumbnails from an external source into Digikam;
> - get the external programs to provide (access to) those thumbnails in a
> well- defined way;
> - find a way to make sure Digikam won't import unchanged thumbnails a second
> time.
> 
> Remco