[Nepomuk] [RFC] New File Indexer

Vishesh Handa me at vhanda.in
Tue Sep 11 15:49:03 UTC 2012


On Tue, Sep 11, 2012 at 8:18 PM, Sebastian Trüg <sebastian at trueg.de> wrote:

> I like this.
>

Good. Then I'll start working on this.


> But I would vote for a plugin system nonetheless. A simple one though. A
> plugin can register for one or more mimetypes and then it simply gets the
> file path and returns a SimpleResourceGraph. You merge all and are done.
> Plugins should never deal with file size, mimetype, or any of those basic
> things the framework can handle.
>

Of course. No point duplicating code.


> This means that the first sweep is done without plugins, the second one
> would call the plugins and the third one, well, that could be yet another
> plugin system which does use RDF types instead of mimetypes. For example:
> the TV show plugin handles nfo:Video. The framework thus calls the plugin,
> provides the path and a handle to the existing metadata. The plugin can
> then simply run its filename analysis and continue from there.
>

For now I'm just going to have the first two steps, but the data will be
saved in one go. And I'll implement all the plugins, after that we can see
about splitting it up into multiple phases like you and Alex have described.

This approach of having multiple steps will also help with the scheduling.
We can do the basic indexing (stat the file + url + mimetype) instantly,
and the rest can be scheduled based on the system usage. ( For people who
don't know - The current approach is to have a queue, whose delay is
controlled based on the system usage )


> OK, one issue we have here is the following: the tv show extractor for
> example works better when run on sets of video files, preferably a whole
> season. Then it only needs to get feedback from the user once or can even
> do its job automatically. This, however, means that third-sweep plugins
> would need an option "can-handle-more-than-one-file-at-a-time".
>
> My 2cents.
>
>
> On 09/11/2012 04:06 PM, Alex Fiestas wrote:
>
>> I think we've discussed this somewhere but I don't remember the outcome
>> of the
>> discussion xD
>>
>> I think that would be really interesting to have an indexer that does a
>> 2pass
>> strategy.
>>
>> First pass will index only basic data such a name, dates, mimetype.
>>
>> Second pass will index specific stuff, previews, texts, tags...
>>
>> Doing this, we can even add third party "information fetchers" as a 3
>> pass,
>> for example to get information about tv shows and such.
>>
>> Let's put an example:
>>
>> -New file in my Downlaod folder detected
>> -Quick super fast indexer indexs data, name, mimetype
>>           From this point, this file is already usable in Nepomuk
>> -Second pass, indexing tags, previews
>> -Third pass (this can be onDemand via GUI) information from the internetz
>> is
>> fetched.
>>
>
+1

Seems like a good idea.


>
>> I got this idea from spotlight (osx indexer metadata thing), the most
>> obvious
>> way of seeing this in osx is when a new external storage is plugged, files
>> will get indexed super fast but all you will get if you perform a search
>> is
>> going ot be filenames, not even mimetypes !
>>
>> Cheerz.
>> ______________________________**_________________
>> Nepomuk mailing list
>> Nepomuk at kde.org
>> https://mail.kde.org/mailman/**listinfo/nepomuk<https://mail.kde.org/mailman/listinfo/nepomuk>
>>
>>  ______________________________**_________________
> Nepomuk mailing list
> Nepomuk at kde.org
> https://mail.kde.org/mailman/**listinfo/nepomuk<https://mail.kde.org/mailman/listinfo/nepomuk>
>



-- 
Vishesh Handa
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/nepomuk/attachments/20120911/ed47a730/attachment.html>


More information about the Nepomuk mailing list