patch for new feature: acoustic fingerprinting and audio similarity

[Jeff Mitchell]

On Tuesday 12 August 2008, Jeff Mitchell wrote:
> On Tuesday 12 August 2008, Soren Harward wrote:
> > Attached is a patch (against SVN-832014) which adds acoustic
> > fingerprinting functionality to Amarok 2.
>
> So it's MusicBrainz?
>
> > Unlike the new UniqueId
> > functionality
>
> It isn't new, it's just ported from 1.4.
>
> > Oh, and can someone approve my SVN commit account on the KDE
> > development site?  I filled out the application on sysadmin.kde.org a
> > couple hours ago.  Thanks.
>
> Usually you don't just start comitting...you send patches to the team until
> they are comfortable with your code and give the OK for you to commit
> directly.
>
> --Jeff

I have many issues with this patch.  The major ones:

-- First and foremost, it will not be merged into Amarok until at least after 
2.0.  We are in feature freeze -- the only reason AFT was ported now was 
because it (the unique ids, at least) was necessary for one of the SoC 
students.  If you want to help with Amarok, the better idea would be to try 
to help us squash bugs and finish the features that already exist...we're on 
a tight timescale.

-- Without the main fingerprinting component working, it is as good as 
vaporware.  That is not a dig at you to say you won't complete it, but rather 
that even without feature freeze, there is no point heavily integrating 
something into every facet of Amarok that doesn't yet exist and that we don't 
know will ever exist.

-- Please provide some use cases.  You must have specific things in mind -- 
what are they?  I suppose it could be used to drive an engine to suggest 
similar tracks from libraries in a local way, which could be useful if the 
person has no access to Last.fm.  But for a very Internet-oriented 
application, that seems like a stretch.  Maybe it's because it's late and I'm 
tired, but I don't really see any good ones popping up in my head.  The only 
other thing I can think of is that it could possibly try to find duplicate 
tracks, but unless it's extraordinarily precise, I doubt it.  (I've seen 
attempts at that before, such as with mp3tunes once you upload tracks to your 
locker -- I have yet to see it work well.)

-- It seems like a duplication of effort.  Music fingerprinting is commonplace 
these days, with MusicBrainz and Last.fm both doing it, and Amarok 1.4 had 
the Moodbar, which essentially did exactly this.  There's also the PUIDs from 
MusicIP, although the only ones that can actually use that data for this type 
of thing is MusicIP itself.  What I'm getting at though is that this kind of 
fingerprinting would belong in a well-documented library so that other apps 
could use it for correlation, not just Amarok, and which could then be 
integrated into Amarok if appropriate.  And it would have to do something to 
separate it from the existing fingerprinting implementations that have 
millions/tens of millions/hundreds of millions of entries to help with 
correlation (if correlation is possible with them, which I'm not sure).

-- If you did end up doing fingerprinting on the audio data, you'll need to 
access all the audio data, probably without any tag information mixed in that 
could throw it off.  This is simple with MP3s; it can be far less simple with 
i.e. Ogg files.  Since we use TagLib, you may want to consider extending 
TagLib (hopefully with Scott's official blessing) to provide a method that 
returns only audio data with no tag data, a kind of the inverse of 
render() -- this would be useful to other people too.

Other, less overarching quibbles:

-- You are linking fingerprinting into every browser and collection, even the 
ones that have no hope of actually using it (i.e. streaming ones, unless you 
start stockpiling a massive database of fingerprints of tracks, intercepting 
audio data as it's streamed).

-- In SqlTrack::calcFingerprint(), you keep making new Fingerprint::Maker 
objects, when it would be far better to keep one lying around and simply pass 
in appropriate tracks.

-- You put fingerprint stuff into Meta::Track, where it doesn't belong, as 
it's specialized to specific types of tracks.

-- You appear to calculate the fingerprint for a track when the track is first 
accessed.  Although you are caching it in the database, this is likely to be 
a very long process.  So forcing it on the user is a no-no.  Note that AFT 
only reads about 16K of a file...I would imagine you would need to read the 
entire thing.  Think about collections stored on home networks accessed (and 
scanned) over WiFi.

--Jeff