D28932: Store filename terms just once

[Stefan Brüns]

bruns created this revision.
bruns added reviewers: Baloo, ngraham.
Herald added projects: Frameworks, Baloo.
Herald added a subscriber: kde-frameworks-devel.
bruns requested review of this revision.

REVISION SUMMARY
  Filename terms where stored twice, once with the "F" filename property
  prefix, and once without prefix. This allows to trivially search for
  files where a term matches in filename or content, but has a number
  of drawbacks:
  
  1. It is not possible to search for a term in content only
  2. The storage size for filenames is approximately doubled
  3. File renaming can cause significant I/O load
  4. Terms appearing in both content and filename may be stored incomplete in the phrase storage.
  
  Re (2.), in case full text indexing is disabled this is a significant
  part of the storage size. With full text indexing, the space savings
  are likely neglegible.
  
  Re (3.), when renaming a file where part of the filename is a common term,
  e.g. "The fox.txt", renaming caused rewriting of data for "the", "fox"
  and "txt". While for "txt" and "fox" this is neglegible, "the" is common
  enough to cause a of rewrite of 10% of the whole DB.
  
  The default search behaviour of matching both filename and content
  can be restored by internally creating queries for both filename and
  content and ORing both together. This extra step does not have any
  noticeable (or even measurable) performance impact.
  
  Depends on D28929 <https://phabricator.kde.org/D28929>

TEST PLAN
  $> ctest -R querytest
  $> baloosearch content:pdf
  $> baloosearch filename:pdf
  $> baloosearch pdf
  $> baloosearch content:pdf OR filename:pdf
  (the last two queries are equivalent)

REPOSITORY
  R293 Baloo

BRANCH
  submit

REVISION DETAIL
  https://phabricator.kde.org/D28932

AFFECTED FILES
  autotests/integration/querytest.cpp
  src/engine/termgenerator.cpp
  src/engine/termgenerator.h
  src/file/basicindexingjob.cpp
  src/lib/searchstore.cpp

To: bruns, #baloo, ngraham
Cc: kde-frameworks-devel, hurikhan77, lots0logs, LeGast00n, cblack, fbampaloukas, domson, ashaposhnikov, michaelh, astippich, spoorun, ngraham, bruns, abrahams
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-frameworks-devel/attachments/20200417/daa03df7/attachment-0001.html>