[Nepomuk] Review Request 112791: Replacing the regexp-based user query parser with a new one (GSoC 2013)

[Denis Steckelmacher]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://git.reviewboard.kde.org/r/112791/
-----------------------------------------------------------

Review request for Nepomuk.


Description
-------

This is the review request for my GSoC 2013 project : A New Query Parser and Auto-Completed Input Widget for Nepomuk.

This big patch removes the old user query parser of Nepomuk (that uses regular expressions to parse a simple language) and adds the query parser developed during the summer. The new query parser uses "parsing passes" that match patterns in the user query and replace them with parsed information. For instance, the parser recognizes that "Foo size > 2M" contains "size > <size>" and that "2M" can be parsed as "<number> <unit>". The resulting query therefore looks for files containing "Foo" and having a size greater that 2 megabytes.

The patterns can be translated using standard KDE localization tools. The patch adds a Messages.sh file in libnepomukcore/ to allow the translators to extract the translations. If this file is not required, it can be removed from the patch.

You can find more information about the internals of the parser on my blog (the archives are here : http://steckdenis.be/archives.html , just look for articles around the middle of July that speak about the parser).

The code added by this patch also lives in the steckdenis-gsoc2013 branch of kde:nepomuk-core. You can find here the whole history of the parser.


Diffs
-----

  CMakeLists.txt 36438aa 
  autotests/test/queryparsertest.h 3c6a833 
  autotests/test/queryparsertest.cpp b63acb3 
  includes/CMakeLists.txt 3ee318e 
  includes/Query/CompletionProposal PRE-CREATION 
  libnepomukcore/CMakeLists.txt 5e9fa86 
  libnepomukcore/Messages.sh PRE-CREATION 
  libnepomukcore/query/queryparser.h ddea63a 
  libnepomukcore/query/queryparser.cpp 80c9bed 
  libnepomukcore/query/queryserializer.cpp 7ac6131 
  libnepomukcore/query/term.h 114baa3 
  libnepomukcore/query/term.cpp cfb41f4 
  libnepomukcore/query/term_p.h 6d92b48 
  libnepomukcore/queryparser/.gitignore PRE-CREATION 
  libnepomukcore/queryparser/completionproposal.h PRE-CREATION 
  libnepomukcore/queryparser/completionproposal.cpp PRE-CREATION 
  libnepomukcore/queryparser/pass_comparators.h PRE-CREATION 
  libnepomukcore/queryparser/pass_comparators.cpp PRE-CREATION 
  libnepomukcore/queryparser/pass_dateperiods.h PRE-CREATION 
  libnepomukcore/queryparser/pass_dateperiods.cpp PRE-CREATION 
  libnepomukcore/queryparser/pass_datevalues.h PRE-CREATION 
  libnepomukcore/queryparser/pass_datevalues.cpp PRE-CREATION 
  libnepomukcore/queryparser/pass_decimalvalues.h PRE-CREATION 
  libnepomukcore/queryparser/pass_decimalvalues.cpp PRE-CREATION 
  libnepomukcore/queryparser/pass_filenames.h PRE-CREATION 
  libnepomukcore/queryparser/pass_filenames.cpp PRE-CREATION 
  libnepomukcore/queryparser/pass_filesize.h PRE-CREATION 
  libnepomukcore/queryparser/pass_filesize.cpp PRE-CREATION 
  libnepomukcore/queryparser/pass_numbers.h PRE-CREATION 
  libnepomukcore/queryparser/pass_numbers.cpp PRE-CREATION 
  libnepomukcore/queryparser/pass_periodnames.h PRE-CREATION 
  libnepomukcore/queryparser/pass_periodnames.cpp PRE-CREATION 
  libnepomukcore/queryparser/pass_properties.h PRE-CREATION 
  libnepomukcore/queryparser/pass_properties.cpp PRE-CREATION 
  libnepomukcore/queryparser/pass_splitunits.h PRE-CREATION 
  libnepomukcore/queryparser/pass_splitunits.cpp PRE-CREATION 
  libnepomukcore/queryparser/pass_subqueries.h PRE-CREATION 
  libnepomukcore/queryparser/pass_subqueries.cpp PRE-CREATION 
  libnepomukcore/queryparser/pass_typehints.h PRE-CREATION 
  libnepomukcore/queryparser/pass_typehints.cpp PRE-CREATION 
  libnepomukcore/queryparser/patternmatcher.h PRE-CREATION 
  libnepomukcore/queryparser/patternmatcher.cpp PRE-CREATION 
  libnepomukcore/queryparser/queryparser.h PRE-CREATION 
  libnepomukcore/queryparser/queryparser.cpp PRE-CREATION 
  libnepomukcore/queryparser/utils.h PRE-CREATION 
  libnepomukcore/queryparser/utils.cpp PRE-CREATION 
  services/fileindexer/indexer/mobipocket/mobipocket.cpp fedc50c 
  services/storage/query/queryservice.cpp 593abd9 
  tests/querytester.h 0f8202d 
  tests/querytester.cpp 65bc454 
  tools/nepomuksearch/main.cpp 931e73d 

Diff: http://git.reviewboard.kde.org/r/112791/diff/


Testing
-------

This patch adds unit tests for the new query parser. These tests verify that the basic English syntax of the new query parser works, and that more advanced features like human-friendly date-time parsing works as expected. They are not complete though, as there are many patterns and rules to test, and in potentially many languages.

The new query parser is fully binary compatible with the old one. I installed my version of nepomuk-core on my openSUSE 12.3 and everything worked, KRunner was able to show me the results of my queries.


Thanks,

Denis Steckelmacher

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/nepomuk/attachments/20130918/6ca31ec1/attachment.html>