Acoustic Fingerprinting and QueryMaker (Bart and Max: read this)

[Bart Cerneels]

On Sun, Jul 12, 2009 at 6:41 PM, Maximilian
Kossick<maximilian.kossick at googlemail.com> wrote:
> On Sat, Jul 11, 2009 at 1:59 PM, Bart Cerneels<bart.cerneels at gmail.com> wrote:
>> On Fri, Jul 10, 2009 at 12:18 PM, Soren Harward<stharward at gmail.com> wrote:
>>> Bart and Max raised issues with the way that the acoustic
>>> fingerprinting patch affects the QueryMaker, which needs to be figured
>>> out before I commit the patch.  I'm moving the discussion here because
>>> I find it easier to handle discussions on the mailing list than on
>>> Fisheye.  So if you don't care about the acoustic fingerprinting
>>> patch, feel free to ignore this thread.
>>>
>>> The acoustic fingerprinting system needs to provide two "use cases"
>>> for programmers:
>>>
>>> 1: Given two tracks, determine how similar one is to another.
>>> 2: Given a track, find other tracks in the collection that are similar to it.
>>>
>>> Use case #1 is currently handled in the Fingerprint class, called by
>>> way of FingerprintCapability, and implemented only in SqlMeta because
>>> local database collections are the only collections capable of storing
>>> fingerprints (as of right now; if Nepomuk collections ever become a
>>> reality, I'll add the functionality there).  Other than some
>>> sloppiness with the database schema which has been fixed, nobody seems
>>> to have an issue with how I have implemented this.
>>>
>>> Use case #2 is currently handled by QueryMaker.  More precisely,
>>> there's a "do nothing" function stub in QueryMaker, and only
>>> SqlQueryMaker overrides this stub to implement it.  Bart and Max,
>>> contrary to what both of you stated in your comments, adding a
>>> function stub to the QueryMaker base class neither adds a dependency
>>> on Fingerprint.h to the base class and all its children, nor does it
>>> require anyone who subclasses QueryMaker to implement the
>>> addSimilarityFilter function.
>>>
>>> I have designed the modifications to be as low-impact as I possibly
>>> could, and I don't know how they could have less impact without
>>> removing the functionality altogether.  It doesn't add a significant
>>> burden on anyone who wants to write a new QueryMaker for Amarok,
>>> because either the collection does not support fingerprinting, in
>>> which case you just ignore the function and let the base class do its
>>> job, or the collection does support fingerprinting, in which case
>>> you're going to have to write fingerprint-handling code anyway.
>>>
>>> Removing the filter from QueryMaker altogether would prove more
>>> complicated that it's worth because then any code that wanted to find
>>> similar tracks would then have to check whether the collection
>>> supports fingerprinting.  So you'd basically end up with a
>>> Capabilities interface for collections, which I think would be far
>>> more invasive than the QueryMaker change.
>>>
>>> The alternative would be to add something like a "tracksSimilarToMe"
>>> function to FingerprintCapability.  This makes less sense to me,
>>> though.  "Find tracks like this one" is a QUERY of the collection, so
>>> conceptually it belongs with the rest of the collection queries (ie,
>>> in QueryMaker).  This approach would also make the Fingerprint class
>>> vastly more complicated, plus slow down the queries significantly.
>>>
>>> --
>>> Soren Harward
>>
>> I don't think you need a collection capability at all. At the moment
>> the fingerprint query is only available in SqlCollection. As long as
>> there is no 2nd implementation there are way better ways to do it. See
>> below.
>> Collection capability exist by the way, though they are a bit hacky.
>
> Well, a bit of foresight might be helpful here in the long run.
> Although it is only implemented in SqlCollection now we should strive
> for an architecture that would allow for easy extension to other
> sources of fingerprints (hey nikolaj, why don't you add the
> fingerprints to the magnatune XML files at some point?), without
> having to go through some major refactoring to make it usable for more
> then one collection.
>
>> I think we can come up with a solution that is a lot cleaner using
>> Bias/Constrained or "whatever is equivalent in your framework". I'll
>> use Bias in this example.
>> From the user perspective you would see a Bias "Add similar sounding
>> tracks from the local collection". The SimilarFingerprintBias is
>> implemented in the SQL plugin and registered with the DynamicPlaylist
>> system.
>> So the query can happen directly in SqlCollection.cpp, you don't even
>> need to use the QueryMaker unless it would be easier to implement.
>
> Well a Bias/Contrained is a given. After all, how would you want to
> get at the similar tracks otherwise? As mentioned in another e-mail,
> there is no point in making it sql specific. Just look at the code in
> Soren's patch! The only SQL specific part of the whole patch is that
> it stores and reads the fingerprint fom the TRACKS table. Everything
> else is generic code that should be implemented once and then be
> immediately available to any collections that starts providing
> fingerprints for its tracks (e.g. Nikolaj could add a fingerprint for
> each magnatune track, or we start writing fingerprints to media
> devices at some point so that they are available when connecting them
> to another computer with Amarok). There isn't any "querying" to speak
> of: the code just grabs a bunch of tracks and basically hopes that one
> of those is similar to the input track.
>
>> Once there is a second collection that implemented fingerprints you
>> can think of creating that capability. That would mean that the
>> Dynamic Playlist system already supports selecting the source of the
>> tracks. I do also have the suspension that those implementations  will
>> inevitably have to depend on GTACFeat, since there is AFAIK no
>> standard format for audio fingerprints or it's not useful in this
>> context.
>
> Can we make it easier for new collections to provide infrastructure by
> implementing the necessary infrastructure correctly right now?
>
> Any part of Amarok that *generates* fingerprints has to depend on the
> fingerprinting stuff. I disagree that just *storing* fingerprint data
> should make that collection depend on GTACFeat. Please compare the
> implementation of Replaygain to this: the replaygain data is stored in
> SqlCollection, and might be stored somehow in other collections too,
> but the code that *uses* that data is in EngineController, and works
> regardless where the track that provides the gain data comes from.
>
>> The FingerprintCapability on Track makes sense, I have no issue there.

Contrary to replaygain though, I don't think there is a standard
format for audio fingerprints. It's not as easy as a single number.
Although I don't think Soren would mind i don't want an Amarok
framework that depends entirely on the data the only one specific
library can generate.

I hope that I'm wrong about the standardized format though. Would be
awsom if we could use the musicbrainz fingerprints for (almost) every
track out there, even the stuff not on our local disk.

Bart