Incremental scanning

Bart Cerneels bart.cerneels at kde.org
Mon Feb 6 09:20:29 UTC 2012


On Wed, Feb 1, 2012 at 14:31, Ville Ranki <ville.ranki at iki.fi> wrote:
>
> Hello,
>
> I've been implementing incremental scanning for Amarok.
>
> Reason for this is that with large connection on a slow
> network share (sftp over ADSL for example) scanning can
> take hours. Also currently all scanned files are lost if
> scanning is interrupted. Even if it would takes few minutes
> to scan entire collection, it is good usability to be
> able to start playing something as soon as Amarok is
> started.

You probably should look for a better solution then mounting over an
ISP connection. I recommend using Ampache or ownCloud. Both should
soon be promoted to fully supported Collections, but can already be
used from the internet services.

>
> I have modified amarokcollectionscanner to output one
> XML block for each scanned directory and ScanManager
> to parse this data on the fly.
>
> Everything works well up to this point when i give
> Directory instances to result processor. In this
> example i have 2 directories with files in database.
>
> (scanner scans first directory)
>
> Following errors are outputted for each file:
>
> [WARNING] [SqlScanResultProcessor] Found urls entry without directory. A
> phantom track. Removing
> "amarok-sqltrackuid://1ca15c03e1fe38d324e128f81afc39a0"
> amarok:     [SqlScanResultProcessor] deleteTrack
> "amarok-sqltrackuid://1ca15c03e1fe38d324e128f81afc39a0" url id 107
> amarok:     [WARNING] [MountPointManager] Device  0  not in database,
> this should never happen!
>
> (second directory is scanned)
>
> For each track:
>  [SqlScanResultProcessor] deleteTrack
> "amarok-sqltrackuid://a15cea27a3d60e37bcee8493e5efcbec" url id 101
>
> In GUI only second directory is visible. Documentation
> on SqlScanResultProcessor is a bit vague. I suppose
> i am using it wrong.
>
> The following is done for each directory. I understood that
> ScanResultProcessor shouldn't be re-used so it's instantieted
> for each directory.
>
> ScanResultProcessor *processor = m_collection->getNewScanResultProcessor();
> processor->setType( m_scanType );
>
> CollectionScanner::Directory *dir = new CollectionScanner::Directory(
> &blockReader );
> processor->addDirectory( dir );
> processor->commit();
> delete processor;
>
> Any ideas what might be wrong?
>
> --

I've recently used the collectionscanner for the USB mass storgage
plugin and scratched my head at ScanRersultProcess myself. Which is
why I ended up not using it.
As far as I can tell the reason it does not do true incremental
scanning is atomicity of the scanning operation. Either everything is
applied or the entire scan result is rolled back. I wonder if that
last case actually happens enough for us to have such a complex system
however.

I can't help you directly with the specific errors you are seeing.
I'll leave that up to the scanner experts (Jeff, Ralf?).

If you are willing to invest a bit more time in this I propose you
look at how collection scanner is used is the UMS plugin and perhaps
use it in a similar way directly from the SqlCollection, without the
ScanResultProcessor overhead.

Bart


More information about the Amarok-devel mailing list