[Nepomuk] [RFC] New File Indexer

[Vishesh Handa]

Hey everyone

Quick update. We have analyzers for -

* taglib
* exiv2
* ffmpeg
* pdf
* plain text files

Documents are still a problem. I've contacted the Calligra team. I'll let
you know what they say.

The analyzers work pretty well. I might just code an epub based analyzer
today.

Tomorrow, I'll start working on a plugin based architecture, and adding two
queues in the index scheduler. One which will immediately call the
SimpleIndexer to just save the basic metadata, and the other one will only
work when on idle. It'll do the proper indexing for the file.

The obvious problem to this approach is that we need a way of saying that
this file has passed the first indexing level, and needs to go through the
second level. Maybe a new property for that?


On Tue, Sep 11, 2012 at 8:18 PM, Sebastian Trüg <sebastian at trueg.de> wrote:

> I like this.
> But I would vote for a plugin system nonetheless. A simple one though. A
> plugin can register for one or more mimetypes and then it simply gets the
> file path and returns a SimpleResourceGraph. You merge all and are done.
> Plugins should never deal with file size, mimetype, or any of those basic
> things the framework can handle.
>
> This means that the first sweep is done without plugins, the second one
> would call the plugins and the third one, well, that could be yet another
> plugin system which does use RDF types instead of mimetypes. For example:
> the TV show plugin handles nfo:Video. The framework thus calls the plugin,
> provides the path and a handle to the existing metadata. The plugin can
> then simply run its filename analysis and continue from there.
>
> OK, one issue we have here is the following: the tv show extractor for
> example works better when run on sets of video files, preferably a whole
> season. Then it only needs to get feedback from the user once or can even
> do its job automatically. This, however, means that third-sweep plugins
> would need an option "can-handle-more-than-one-**file-at-a-time".
>
> My 2cents.
>
>
> On 09/11/2012 04:06 PM, Alex Fiestas wrote:
>
>> I think we've discussed this somewhere but I don't remember the outcome
>> of the
>> discussion xD
>>
>> I think that would be really interesting to have an indexer that does a
>> 2pass
>> strategy.
>>
>> First pass will index only basic data such a name, dates, mimetype.
>>
>> Second pass will index specific stuff, previews, texts, tags...
>>
>> Doing this, we can even add third party "information fetchers" as a 3
>> pass,
>> for example to get information about tv shows and such.
>>
>> Let's put an example:
>>
>> -New file in my Downlaod folder detected
>> -Quick super fast indexer indexs data, name, mimetype
>>           From this point, this file is already usable in Nepomuk
>> -Second pass, indexing tags, previews
>> -Third pass (this can be onDemand via GUI) information from the internetz
>> is
>> fetched.
>>
>> I got this idea from spotlight (osx indexer metadata thing), the most
>> obvious
>> way of seeing this in osx is when a new external storage is plugged, files
>> will get indexed super fast but all you will get if you perform a search
>> is
>> going ot be filenames, not even mimetypes !
>>
>> Cheerz.
>> ______________________________**_________________
>> Nepomuk mailing list
>> Nepomuk at kde.org
>> https://mail.kde.org/mailman/**listinfo/nepomuk<https://mail.kde.org/mailman/listinfo/nepomuk>
>>
>>  ______________________________**_________________
> Nepomuk mailing list
> Nepomuk at kde.org
> https://mail.kde.org/mailman/**listinfo/nepomuk<https://mail.kde.org/mailman/listinfo/nepomuk>
>



-- 
Vishesh Handa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/nepomuk/attachments/20120912/d50f3d40/attachment.html>