Database separation for similarity searches

[Mario Frank]

Hey Gilles,

I completed the adoptions yesterday and just pushed the changes to the
db-sep branch.

I tried to describe still necessary or potentially necessary changes in
the commit message.

As I am no specialist in MySQL, especially the dbconfig file should be
revised by someone who

is more experienced with MySQL. Also, the Core DB schema will have to be
adopted in a new revision

as the ImageHaarMatrix is not used in CoreDB anymore.

I tested the changes with SQLite which works fine.

Here is the commit, sadly, quite large commit:

https://cgit.kde.org/digikam.git/commit/?h=gsoc17-db-sep&id=e4dc5b9260ad771ed1dbae33c65c41eb50906b3e


Now that we have a separate similarity database, the fuzzy search can be
boosted enormously.

For this, I have to store the calculated similarities in the database so
I can reuse them.

So, there will be some adoptions on haariface for 6.0.0.

Cheers,

Mario


Am 25.01.2018 um 17:46 schrieb Gilles Caulier:
> Ok,
>
> I just sync the branch with master. At least, the branch is up to
> date, as 6.0.0. this will be more easy to merge in 6.0.0 later.
>
> Gilles
>
> 2018-01-25 17:08 GMT+01:00 Mario Frank <mafrank at uni-potsdam.de
> <mailto:mafrank at uni-potsdam.de>>:
>
>     Hey Gilles,
>
>     I got the code working finally, yesterday evening. It now compiles
>     but the similarity db is not yet initialised correctly on startup.
>
>     There were far more changes and extensions necessary than I
>     expected. So far about the current status.
>
>     Mario
>
>
>     Am 22.01.2018 um 19:51 schrieb Gilles Caulier:
>>     Hi Mario,
>>
>>     for digiKam 6.0.0 definitively. I sync the branch with master
>>     step by step. It's fine to merge db-sep branch to 6.0.0 branch
>>     that i created 2 week ago.
>>
>>     Gilles
>>
>>     2018-01-22 16:55 GMT+01:00 Mario Frank <mafrank at uni-potsdam.de
>>     <mailto:mafrank at uni-potsdam.de>>:
>>
>>         Hey Gilles,
>>
>>         I looked at the code yesterday and it is not yet finished. I
>>         will apply some changes to improve performance and report on
>>         it before committing.
>>
>>         For which release is the db separation planned? 5.9 or 6.0?
>>
>>         Best
>>
>>         Mario
>>
>>
>>         Am 06.01.2018 um 17:43 schrieb Mario Frank:
>>>         Hey Gilles,
>>>
>>>         I think she was not completely finished. I will have time
>>>         for digiKam again after this weekend. I will take a look.
>>>         I've got unfinished work anyway.
>>>
>>>         Cheers
>>>         Mario
>>>
>>>         Am 6. Januar 2018 16:16:37 MEZ schrieb Gilles Caulier
>>>         <caulier.gilles at gmail.com> <mailto:caulier.gilles at gmail.com>:
>>>
>>>             Note : with this commit, i sync this branch with master
>>>             without conflict.
>>>
>>>             https://commits.kde.org/digikam/d8511b14fd3e9c3406357858f557bdefa03d1551
>>>             <https://commits.kde.org/digikam/d8511b14fd3e9c3406357858f557bdefa03d1551>
>>>
>>>             Gilles
>>>
>>>             2018-01-06 16:13 GMT+01:00 Gilles Caulier
>>>             <caulier.gilles at gmail.com
>>>             <mailto:caulier.gilles at gmail.com>>:
>>>
>>>                 Hi all,
>>>
>>>                 This summer, Swaty has work on database separation
>>>                 for similarity searches. The goal was to separate
>>>                 the fuzzy fingerprints storage from core DB to a
>>>                 dedicated DB. This will simplify the backup and
>>>                 prevent to overload the core DB with large data by
>>>                 image (around 500 bytes by fingerprints)
>>>
>>>                 The work is not completed as i know. All have be
>>>                 done in origin/gsoc17-db-sep branch from git repository.
>>>
>>>                 What's the status of this code exactly ? Can we
>>>                 continue with this implementation, or it will be
>>>                 better to re-start this work from scratch ?
>>>
>>>                 The DB separation is important for the future. Thank
>>>                 sin advance for your feedback.
>>>
>>>                 Best
>>>
>>>                 Gilles
>>>
>>>
>>>
>>>         -- 
>>>         Diese Nachricht wurde von meinem Android-Mobiltelefon mit
>>>         K-9 Mail gesendet. 
>>
>>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/digikam-devel/attachments/20180126/c204ca45/attachment.html>