Problems with sqlite support with 1.4.8: no such table: uniqueid_temp

[Jeff Mitchell]

Hey, you jogged my memory.  Now I know why you get the errors about _temp, 
although I still have no answers for you about your CPU issue.

On Monday 25 February 2008, Colin Guthrie wrote:
> I think it has something to do with playlists.

It could have something to do with the main Amarok Playlist, but other than 
that none of them use AFT.

So here's why you get those _temp errors.

You said you were doing full rescans.  When the collection scanner is doing an 
incremental scan it creates temporary tables populated with old information; 
then it drops the normal tables, scans, and copies the info from the temp 
tables back (I think I have that sequence right).  However, when doing a full 
rescan it drops all tables and just scans directly in, it doesn't create 
temporary tables.  So when the unique ID for a URL is wanted, the function 
checks to see if scanning is currently happening.  If so, it defaults to 
looking up the UID in the temp table (because the scan isn't yet complete).  
If it gets no value back, it then checks the normal table.  Since Amarok 
prints out these errors from the database what you're seeing is simply a 
result of checking the temp table first, which may not exist.  It's a totally 
harmless error.  The reason it checks the temp tables first is because it if 
you have information about a unique ID in both locations it defaults to the 
old information in case they've not yet gotten merged.  That was actually 
based on an old architecture of AFT and is probably not relevant 
anymore...rethinking the issue, I don't see why it couldn't be flipped.  
Although you could still see the errors if you're scanning and the new 
database tables do not yet have that URL value in them (so a better way is if 
you know a backend-agnostic query to find out whether or not a table 
exists...).

> As you are rebuilding it, the scanner runs for a long time. It finds
> playlists and populates these in the system. Meanwhile the
> contextbrowser  may try and refresh the it's files in some capacity
> (perhaps due to the fact playlists are found/updated?) which it does via
> collectiondb instance.

No, whatever issue you're seeing, it definitely has nothing to do with the 
contextbrowser (or non-Playlist playlists for that matter).  Neither have 
anything to do with AFT.

> However, perhaps the contextbrowser runs in a different thread than the
> scanner and thus cannot access the temporary tables? In
> CollectionDB::uniqueIdFromUrl() it detects whether it's scanning based
> via a way that *is* threadsafe.

Well, should be threasafe, yes, as the scan controller runs in the GUI thread 
(although the collection scanner does not).

> So the contextbrowser which cannot 
> access the temp tables thinks it is currently scanning and tries to
> access them?

No, but if you are e.g. adding items to the Playlist during the scan, it will 
read metabundles for those items, causing the uniqueIdForURL function to be 
called.

> I do see a QDeepCopy relating to bundles in playlistbrowseritem.cpp and
> a call to uniqueIdFromUrl() from within metabundle so this looks like a
> possible candidate as IIRC QDeepCopy is not thread safe?

No, because this is all going on in the GUI thread.

> To at least attempt to verify this theory I added a flag to
> CollectionDB::uniqueIdFromUrl() to ignore the scanning detection and
> setup metabundle's call to it to use this flag (the only place where I
> set this flag). Low and behold the scan worked!!!!!!!!!!!!

Well, then, send the code, man.  :-)

Without seeing your code I can't tell you why things might have suddenly 
changed for you.  In fact, I don't know what else you're doing at the time, 
either (do you have items in your Playlist?  Are you just letting Amarok be 
or are you using it in some fashion as you scan?  Are you loading items from 
the Collection Browser into the Playlist as you scan? etc).  All these things 
matter because if you want to propose that it's a threadsafety problem, you 
haven't given enough information to know what threads might be going on at 
the time, other than the collection scanner and the main GUI thread.

Most likely what you've found is probably some sidestepping around whatever 
your real issue is.  Nothing in that function that is not an outside call 
should not be threadsafe.  And I don't know of anything in those outside 
calls (to ScanController and MountPointManager for instance) that shouldn't 
be threadsafe either.

Have you tried replicating this problem on MySQL or PostgreSQL?  For the 
record, I'm currently using Amarok's internal sqlite, and I've tried things 
like adding files to the playlist while a scan is running, and I've not been 
able to run into issues.

--Jeff