Digikam raw files and darktable

[Juan Jose Casafranca]

On sábado, 7 de enero de 2017 13:51:57 (CET) Gilles Caulier wrote:
> 2017-01-07 12:51 GMT+01:00 Juan Jose Casafranca <jjcasmar at gmail.com>:
> > I think you are trying to solve the problem in a very complex way.
> > 
> > First of all, where does digikam currently saves the thumbnails? If it's
> > in
> > the database, then, the only thing needed is a way to generate the
> > processed
> > raw as a thumbnail and a flag which specify if the thumbnail is up to
> > date.
> > 
> > This flag es very easy to maintain. When loading the database, the
> > thumbnails
> > are not up to date. So darktable (or any other raw processor) will be
> > called
> > to generate the thumbnails. When a raw file is opened for edition through
> > the
> > digikam interface, then also mark the thumbnail is not up to date. If the
> > user
> > opens the file for edition from other place different from the digikam
> > interface (opening the raw processor directly for example), then do
> > anything.
> > I know that thumbnail stored and the processed image will not match in
> > this
> > case, this should be documented. Enable an option to update manually the
> > thumbnails for raw files.
> > 
> > Digikam should have an option to choose which raw processor want the user
> > to
> > user. Default one, darktable, raw therapee, whatever. When a user try to
> > edits
> > a raw file, then the raw processor selected is opened.
> > 
> > I dont think that we need to store any extra information in the xmp file.
> > Simply, made different software dont override other software xmp tags (as
> > I
> > said in other post, I think this is common sense).
> > 
> > Digikam would not treat darktable in a special way. I'm just asking for an
> > interface to connect digikam with any raw processor. This interface should
> > only need to provide a way to open the raw processor with the selected
> > image
> > and a way to get back the image from the raw processor.
> > 
> > I dont really understand why digikam and raw processors should communicate
> > through the xmp file.
> > 
> > So, to summarize:
> > User selects darktable for raw editing
> 
> Already possible in Linux desktop, through OpenWith context menu.
> 
> > User opens a raw file in digikam -> darktable is opened
> 
> Already possible in Linux Desktop, with default application set in desktop,
> that can be handle through a keyboard shortcut. I introduced myself in
> 5.0.0. Perhaps this need to be improved, but a good desktop settings in a
> best start.
> 
> > User edits in darktable and close it -> thumbnail is marked as not up to
> > date
> > Thumbnail is updated calling darktable which returns an image (maybe with
> > a
> > LUA script?)
> 
> DBUS can be used here. It's easy, but... This will only work under Linux.
> Forget MacOS and Windows, DBUS is a waste of time.

What do you propose for windows and mac? Or just implement this as a linux 
only feature? (Anyway, windows and mac users already has another raw 
processing workflow which includes other software, dont they?)

> 
> > The thumbnail is stored in digikam the same way other jpeg or current raw
> > thumbnails are stored
> 
> On this point, i'm not agree. It"s a regression and an user wish fixed few
> year ago...

How do you propose to store raw thumbnails then? I don't really know how 
digikam stores thumbnails for other types, or even if it stores something or 
simply generate them online.

> 
> Gilles Caulier
> 
> > On sábado, 7 de enero de 2017 12:33:43 (CET) Remco Viëtor wrote:
> > > On samedi 7 janvier 2017 10:31:27 CET Gilles Caulier wrote:
> > > > I follow this thread, and that i can said is : RAW workflow is
> > > > terrific
> > > > and
> > > > very complex. I spare few years to undstand all main subtilities to
> > > > implement a first suitbale support of RAW files in digiKam.
> > > > 
> > > > Read the contextual responses below for the next...
> > > > 
> > > > 2017-01-07 10:15 GMT+01:00 Remco Viëtor <remco.vietor at wanadoo.fr>:
> > > > > On samedi 7 janvier 2017 03:42:35 CET Juan Jose Casafranca wrote:
> > > > > > I know that darktable writes not only tags, rating etc, but also
> > 
> > the
> > 
> > > > > > process history.
> > > > > > 
> > > > > > That's why I want digikan to call darktable when it founds a raw
> > 
> > file.
> > 
> > > > > > (...)
> > > > > 
> > > > > Wouldn't it be easier to get an option in Darktable to write a
> > 
> > thumbnail
> > 
> > > > > file
> > > > > to disk when leaving darkroom mode or changing file there? And then
> > 
> > add
> > 
> > > > > a
> > > > > link
> > > > > to that thumbnail in the XMP file. Basically, you add an API for
> > > > > external
> > > > > programs to provide a thumbnail image (and afaik, KDE has a
> > 
> > standardised
> > 
> > > > > way
> > > > > to store thumbnails in a directory).
> > > > 
> > > > This API is limited even if it's standardized by Open Desktop. digiKam
> > > > drop
> > > > this support since a while for multiple technical reasons, as using
> > > > Wavelets compression to reduce space instead PNG, using a dedicted
> > > > database
> > > > to handle disconnected removable media, be able to store thumb in a
> > 
> > remote
> > 
> > > > database, support more than 256x256 thumbnails size, etc...
> > > > 
> > > > So no KDE API here. In all case, we want to reduce to the max KDE
> > > > dependencies for the future to be porta le everywhere and reduce
> > > > complexity.
> > > > 
> > > > Note : KDE thumbnials use KIO : we drop this support since 5.0, which
> > 
> > DO
> > 
> > > > NOT WORK outside Linux, and cannot be bundled in AppImage. No KIO in
> > > > digiKam for the future...
> > > 
> > > OK, so that is not a viable option. But a clear way for external
> > 
> > programs to
> > 
> > > get information (like thumbnails) back to Digikam would be helpful, and
> > 
> > the
> > 
> > > sidecar could be a channel for that.
> > > 
> > > > > And Darktable already generates preview
> > > > > images/thumbnails (top left in the darkroom window, and of course in
> > > > > lighttable mode). Better to reuse those if at all possible.
> > > > 
> > > > Where are stored this preview image ? In a dedicated place... This
> > 
> > will be
> > 
> > > > another puzzle to interface digiKam with that...
> > > 
> > > Ideally, Digikam shouldn't have to worry about if and where those
> > 
> > thumbnails
> > 
> > > are stored internally by Darktable. But the code to *generate* those
> > > thumbnails is in place already, what's missing is a way to *export*
> > > those
> > > thumbnails for use with external programs (i.e. Digikam).
> > > 
> > > > > It's clear that Digikam's editor will not be able to generate a
> > > > > thumbnail
> > > > > from
> > > > > the original RAW
> > > > 
> > > > Wrong. digiKam use libraw and extract JPEG preview from RAW file to
> > > > generate thumbnails...
> > > > 
> > > > > and a Darktable XMP. But starting Darktable for each and
> > > > > (...)
> > > 
> > > Sorry, you cut my phrase at the wrong point...
> > > What I wanted to say is that Digikam cannot repeat the processing
> > 
> > Darktable
> > 
> > > does, i.e. cannot generate a thumbnail by combining the information from
> > 
> > the
> > 
> > > RAW file and the XMP file (of course Digikam can generate a thumbnail
> > 
> > from
> > 
> > > the RAW file only, it's doing just that all the time...)
> > > 
> > > > A possible solution is to use IPTC field stored in XMP. There is a
> > > > IPTC
> > > > preview tag to host JPEG preview. It's limited to 256Kb of data which
> > 
> > is
> > 
> > > > enough to do the job. Problem : Old IPTC support is, in XMP i don't
> > 
> > know,
> > 
> > > > especially to host binary data in XML.
> > > > 
> > > > Other problem is to explode the sidecar XMP file size. This is why we
> > 
> > use
> > 
> > > > a
> > > > dedicted database....
> > > > 
> > > > > Another problem with that would be all the other RAW processors out
> > > > > there.
> > > > > Why
> > > > > would Digikam treat Darktable in a special way and ignore all the
> > > > > others?
> > > > 
> > > > Remember that we use already a RAW processor : libraw. there are
> > 
> > plenty of
> > 
> > > > new feature in this processor not yet used in digiKam. The best way is
> > 
> > to
> > 
> > > > implment new RAW feature in digiKam, at least to import RAW data in
> > > > editor.
> > > 
> > > Possible, but not relevant to this subject (IMO): there are many reasons
> > 
> > why
> > 
> > > someone would use Digikam for DAM, and another program for RAW
> > 
> > development.
> > 
> > > The point here is to get a *representation* of the image, as produced by
> > > that other program, in the Digikam thumbnails view.
> > > 
> > > So, basically, what I'd aim for is:
> > > - define a way to get thumbnails from an external source into Digikam;
> > > - get the external programs to provide (access to) those thumbnails in a
> > > well- defined way;
> > > - find a way to make sure Digikam won't import unchanged thumbnails a
> > 
> > second
> > 
> > > time.
> > > 
> > > Remco