D11038: balooctl: Add pruneDb option to remove stale file index entries.

[Michael Heidelbach]

michaelh added a comment.


  Baloo is in bad need for something like this. Unfortunately in cannot be done so easily.
  We have to account for indexed net shares and removable drives which are only temporarily unavailable. Files on those should not be removed. And most likely some user interaction is needed here.
  
  #Baloo <https://phabricator.kde.org/tag/baloo/> has it own Project page now. Please have a look and file your plans for Baloo as tasks there. Also I'm very much interested in your opinion about the tasks filed there specially about  T8054 <https://phabricator.kde.org/T8054> and .

INLINE COMMENTS

> transaction.cpp:264
> +    QTextStream out(stdout);
> +    out << "Total Document IDs: " << map.size() << endl;
> +

This should go to stderr so only pruned files go to stdout

> transaction.cpp:270
> +        if (!QFileInfo::exists(url)) {
> +            out << "Purging " << id << endl;
> +            m_writeTrans->removeDocument(id);

Place this at the end of the loop and also print url, please. E.g. `out << "Removed" << id << ":" << url;` would be easily parseable. So one could catch stdout with

  balooctl pruneDb | tr '-d:' -f2 >removed_files.lst

> transaction.cpp:271
> +            out << "Purging " << id << endl;
> +            m_writeTrans->removeDocument(id);
> +            count++;

Can we or should we check the result of the operation here?

> transaction.cpp:276
> +
> +    out << "Removed Entries: " << count << " (" << count * 100.0 / map.size() << "%)" << endl;
> +}

Same as above

> main.cpp:90
>      parser.addPositionalArgument(QStringLiteral("monitor"), i18n("Monitor the file indexer"));
> +    parser.addPositionalArgument(QStringLiteral("pruneDb"), i18n("Purge invalid index entries"));
>      parser.addPositionalArgument(QStringLiteral("indexSize"), i18n("Display the disk space used by index"));

Just 'prune'?

> main.cpp:326
>  
> +    if (command == QStringLiteral("pruneDb")) {
> +        Database *db = globalDatabaseInstance();

see above

> main.cpp:329
> +        if (!db->open(Database::ReadOnlyDatabase)) {
> +            out << "Baloo Index could not be opened\n";
> +            return 1;

stderr

> main.cpp:334
> +        Transaction tr(db, Transaction::ReadOnly);
> +        out << "Checking file paths .. " << endl;
> +        tr.pruneFsTree();

stderr

REPOSITORY
  R293 Baloo

REVISION DETAIL
  https://phabricator.kde.org/D11038

To: smithjd, #baloo, #frameworks
Cc: michaelh, ashaposhnikov, spoorun, nicolasfella, alexeymin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-frameworks-devel/attachments/20180305/b4e2d5fd/attachment-0001.html>