FFMPEG video metadata parser into 6.0.0

[Gilles Caulier]

Hi all,

As you seen certainly, after many years to wait a change from Exiv2 to
support video metadata safety, we lost certainly a chance to see this Exiv2
code reviving in the future.

THE STATE :

- Exiv2 video support is bugous and crash immediately digiKam if
collections has video files.
- i rerouted more than 60 bugs about Exiv2 video crash. There are certainly
more files bugs it's a very long task to analyze GDB backtrace, when it's
available.
- Since 5.3.0, all bundles drop Exiv2 video support to turn off the cmake
option at compilation time.
- Even if video support is disabled in Exiv2, only the video file parser
are not compiled. All XMP multimedia XMP namespace still here and can be
used in RW.

THE SOLUTION :

- As we use QtAv for playing with video, FFMPEG is used in background.
- QtAv do not provide a simple wrapper to extract video metadata, even if
these info can be displayed in file properties from QtAV player. I reported
this whish in QtAV bugzilla, but lead developer estimate this change as out
of scope of the project.
- I add FFMPEG dependencies in digiKam (libav C API). This is not a really
new dependency as we use it indirectly through QtAV. In all case to compile
bundles, we need FFMPEG API installed. So the puzzle is not increased...
- I write a good FFMPEG metadata parser with a Exif/IPTC/XMP wrapper.
The code currently start ffmpeg API (as ffprobe do from CLI) and look for
metadata entry from each streams (main container, video container, audio
container)

https://cgit.kde.org/digikam.git/tree/libs/dmetadata/dmetadata_ffmpeg.cpp?h=development/6.0.0

- The parser is voluntary limited to one video stream and one audio stream.
The standard XMP Multimedia namespaces do not support multiple streams, as
i seen (fix me if i'm wrong here). So while all streams from a video
container are parsed, when first audio and video entry is found, there are
used to populate XMP. Other entries are ignored.

The parser will fix 95% of use cases to populate the Database entries. The
goal is to be able to found video by date, make/model, orientation, codecs,
size, etc... And the best, GPS info are also extracted if information
exists in file. So a search over a map will be possible...

But there is more : internal ffmpeg tags parser is able to get some entries
set by video production softwares, as comments, rating, keywords,
categories, etc. All these information are dispatched everywhere into Exif,
Iptc and Xmp for the pleasure of digiKam database file scanner.

Important : the parser if very fast, as no video data is decoded to extract
metadata. I write 4 CLI unit test tools to show lead info, all Exif, all
Iptc, and all Xmp metadata from CLI (these tools work also with images).
Look loadfromfile_*.cpp files here :

https://cgit.kde.org/digikam.git/tree/tests/dmetadata?h=development/6.0.0

THE FUTURE:

Currently, we have a code to read only the video metadata. What about
writting support ?

I found this kind of code in GitHub :

https://github.com/kritzikratzi/ofxAvCodec/blob/master/src/ofxAvUtils.cpp#L61

This C++ code is a tool to read metadata (poor implementation compared to
mine), and to write. This last part is interesting, and a little bit
complex, and i must admit that i'm not sure to understand well how it work
exactly. I suspect that a new video file is created from an original one
where metadata need to be patched. Stream are backported, and later all
metadata rewritten and mixed from scratch. This can be a time consuming and
not an optimized way, but at least, it must work. FFMPEG do not permit to
play directly with metadata entries, it's not a framework dedicated for
that.

Somebody can help me to decode this method in details, at least to see if
my analyze is not too bad ? Thanks in advance.

LIMITATION:

The current internal FFMPEG video parsers are limited. This depend of video
format, and metadata tags supported. The best video container supported is
MP4 / QuickTime. If you want to see this state improved in the future,
please report the problem  to FFMPEG team who maintain this code. Comparing
FFMPEG code to ExifTool, and also Exiv2 parsers, i see that FFMPEG do not
support yet all entries.

TESTING:

To check at least a large amount of use cases, i will need test video files
tagged with video production tools, to see if the digiKam FFMPEG parser do
the job properly. Using Web cloud services will be the best way to share
files.

Note : here i can test with Final Cut Pro as my son use this software for
video editing. I need Vegas Pro, Premiere, and other test items...

Thanks in advance for your feedback

Gilles Caulier
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/digikam-devel/attachments/20180301/4be42c67/attachment.html>