Review Request 125762: External extractor plugin support for KFileMetaData

[Boudhayan Gupta]

> On Dec. 6, 2015, 5:32 p.m., Vishesh Handa wrote:
> > If you're planning on pushing this please push it to a testing branch. Untill we actually have some plugins using this we might not be sure of the API. My points below are quite negative and I was hoping on writing a more positive email about the different ways to go about this, but I seem to have been procrastinating and have not written it. Sorry.
> > 
> > My main objections with this -
> > 
> > 1. One cannot choose between external plugins. Since all of them are in 1-plugin (called ExternalPlugin), and all the mimtypes are just combined. This matters less in the case of Baloo, but when someone else wants to choose specific extractors. It's impossible.
> > 
> > 2. Extraction of plain text is simply not supported.
> > 
> > 3. No way to differentiate in the implementation of a plugin. We may get plugins for the same mimetype using different technologies, perhaps we need a way to identify which one to choose. Maybe this will never be a problem, but I'm skeptical.
> > 
> > 4. Contamination of the PATH, maybe we could put them in a different place which makes it obvious that users should never execute them.
> > 
> > 5. Getting the list of extractors now means running a possibly large number of executables with possibly bad implementations. They could just get stuck, and all other plugins will suffer. Perhaps the list of mimetypes supported could be in a desktop file?

I'm going to sit and think on this for a while.

I wanted to get a minimally invasive solution to the problem, but if you want it doen **right**, I'd have to think things through.

I don't think 3 will ever be a problem - I'm suggesting that people will install only one plugin per mimetype, if someone wants a plugin using different tech, they'll swap out the current plugin (by removing it from their system). For 4, we can put the somewhere in libexec. For 5, I don't want to use .desktop files. Maybe KPackage based solutions? Or even simpler, a bunch of json files in some subdir of /etc.


- Boudhayan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://git.reviewboard.kde.org/r/125762/#review89174
-----------------------------------------------------------


On Oct. 24, 2015, 5:49 p.m., Boudhayan Gupta wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://git.reviewboard.kde.org/r/125762/
> -----------------------------------------------------------
> 
> (Updated Oct. 24, 2015, 5:49 p.m.)
> 
> 
> Review request for Baloo, KDE Frameworks, Pinak Ahuja, and Vishesh Handa.
> 
> 
> Repository: kfilemetadata
> 
> 
> Description
> -------
> 
> This patch introduces support for external metadata extractors in KFileMetaData
> 
> The external extractors themselves can be written in any language, provided that it can be executed as a standalone executable (compiled or script with a hashbang), with command line arguments, and can output data to stdout.
> 
> The extractors are executed like so:
> 
> * `extractor --mimetypes` - outputs a list of mimetypes supported by the extractor, one per line.
> * `extractor filename` - outputs a json document with the metadata. The keys are such that they can be directly used with PropertyInfo::fromName().
> 
> At the KFileMetaData end, an additional internal plugin (ExternalExtractor) is provided that forms a conduit between external extractors and the internal API. This plugin looks for executables called kfilemetadata_extractor_<something> in /usr/bin to find external extractors, and executes them with the --mimetypes arg to find the list of mimetypes each extractor supports. ExternalExtractor then claims to support all of these mimetypes, and then delegates to the extractor executable when doing the actual extraction.
> 
> 
> Diffs
> -----
> 
>   README.md 19b1a26 
>   src/extractors/CMakeLists.txt 5dd223e 
>   src/extractors/externalextractor.h PRE-CREATION 
>   src/extractors/externalextractor.cpp PRE-CREATION 
> 
> Diff: https://git.reviewboard.kde.org/r/125762/diff/
> 
> 
> Testing
> -------
> 
> Tested with the sample executable file extractor (as attched, written in python) with the dump manual test in KFileMetaData. Works.
> 
> 
> File Attachments
> ----------------
> 
> kfilemetadata_extractor_executable
>   https://git.reviewboard.kde.org/media/uploaded/files/2015/10/23/146b657f-31d9-4117-a82f-ef966a6339d4__kfilemetadata_extractor_executable
> 
> 
> Thanks,
> 
> Boudhayan Gupta
> 
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-frameworks-devel/attachments/20151207/d3f27ee3/attachment-0001.html>