Review Request 114717: Language detection in Sonnet

Wed Jan 8 19:24:50 GMT 2014

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://git.reviewboard.kde.org/r/114717/
-----------------------------------------------------------

(Updated Jan. 8, 2014, 7:24 p.m.)

Status
------

This change has been marked as submitted.

Review request for kdelibs and KDEPIM.

Repository: sonnet

Description
-------

I started by merging in the old language detection branch from SVN, while improving it as I went along. One improvement was to use QChar's unicode information instead of shipping our own unicode code point information tables. The old filter class also got replaced with a new tokenizer, which I rewrote most of to simplify.

I added kdepim to the reviewers because I remember talking with someone working on PIM stuff on IRC, and he was interested in this (a long time ago, though).

Diffs
-----

  data/trigrams/ja PRE-CREATION 
  data/trigrams/kk PRE-CREATION 
  data/trigrams/ko PRE-CREATION 
  data/trigrams/ky PRE-CREATION 
  data/trigrams/la PRE-CREATION 
  data/trigrams/lt PRE-CREATION 
  data/trigrams/lv PRE-CREATION 
  data/trigrams/mk PRE-CREATION 
  data/trigrams/mn PRE-CREATION 
  data/trigrams/nb PRE-CREATION 
  data/trigrams/ne PRE-CREATION 
  data/trigrams/nl PRE-CREATION 
  data/trigrams/nr PRE-CREATION 
  data/trigrams/pl PRE-CREATION 
  data/trigrams/ps PRE-CREATION 
  data/trigrams/pt PRE-CREATION 
  data/trigrams/pt_BR PRE-CREATION 
  data/trigrams/pt_PT PRE-CREATION 
  data/trigrams/ro PRE-CREATION 
  data/trigrams/ru PRE-CREATION 
  data/trigrams/sk PRE-CREATION 
  data/trigrams/sl PRE-CREATION 
  data/trigrams/so PRE-CREATION 
  data/trigrams/sq PRE-CREATION 
  data/trigrams/sr PRE-CREATION 
  data/trigrams/ss PRE-CREATION 
  data/trigrams/st PRE-CREATION 
  data/trigrams/sv PRE-CREATION 
  data/trigrams/sw PRE-CREATION 
  data/trigrams/th PRE-CREATION 
  data/trigrams/tl PRE-CREATION 
  data/trigrams/tn PRE-CREATION 
  data/trigrams/tr PRE-CREATION 
  data/trigrams/ts PRE-CREATION 
  data/trigrams/uk PRE-CREATION 
  data/trigrams/ur PRE-CREATION 
  data/trigrams/uz PRE-CREATION 
  data/trigrams/ve PRE-CREATION 
  data/trigrams/vi PRE-CREATION 
  data/trigrams/xh PRE-CREATION 
  data/trigrams/zu PRE-CREATION 
  sonnet.yaml c54f87b 
  src/CMakeLists.txt e79492f 
  src/core/CMakeLists.txt 2f8a184 
  src/core/backgroundchecker.cpp 8b9e983 
  src/core/backgroundchecker_p.h PRE-CREATION 
  src/core/backgroundengine.cpp 3a14d34 
  src/core/backgroundengine_p.h 10f6a27 
  src/core/client_p.h bd3e416 
  src/core/filter.cpp e99d332 
  src/core/filter_p.h 6c7d8c9 
  src/core/globals.h 0c54c96 
  src/core/globals.cpp e57450f 
  src/core/guesslanguage.h PRE-CREATION 
  src/core/guesslanguage.cpp PRE-CREATION 
  src/core/languagefilter.cpp PRE-CREATION 
  src/core/languagefilter_p.h PRE-CREATION 
  src/core/loader.cpp ee8db0e 
  src/core/settings.cpp 095eddb 
  src/core/settings_p.h ee2d22c 
  src/core/speller.h 7428339 
  src/core/speller.cpp 8cc2a1e 
  src/core/textbreaks.cpp PRE-CREATION 
  src/core/textbreaks_p.h PRE-CREATION 
  src/core/tokenizer.cpp PRE-CREATION 
  src/core/tokenizer_p.h PRE-CREATION 
  src/plugins/CMakeLists.txt fc33a97 
  src/plugins/aspell/kspell_aspellclient.h eadb52a 
  src/plugins/enchant/CMakeLists.txt 817db0c 
  src/plugins/enchant/enchantclient.h 25f62eb 
  src/plugins/hspell/CMakeLists.txt e128cb3 
  src/plugins/hspell/kspell_hspellclient.h 966303f 
  src/plugins/hunspell/CMakeLists.txt ccae7f7 
  src/plugins/hunspell/kspell_hunspellclient.h 79638bb 
  src/ui/configui.ui 6532552 
  src/ui/configwidget.cpp 7a5cc99 
  src/ui/dialog.cpp 13ad39d 
  src/ui/highlighter.h 46418b9 
  src/ui/highlighter.cpp 9f31268 
  src/unicode/CMakeLists.txt 1be0a54 
  src/unicode/README f9b8030 
  src/unicode/data/GraphemeBreakProperty.txt 8805f36 
  src/unicode/data/SentenceBreakProperty.txt fc58820 
  src/unicode/data/WordBreakProperty.txt 78c531c 
  src/unicode/parseucd/parseucd.cpp a050140 
  tests/test_dialog.cpp 0579bb2 
  tests/test_highlighter.h 9cf5657 
  tests/test_highlighter.cpp 695a2df 
  tests/test_textedit.cpp 5c02809 
  data/trigrams/fr PRE-CREATION 
  data/trigrams/ha PRE-CREATION 
  data/trigrams/hi PRE-CREATION 
  data/trigrams/hr PRE-CREATION 
  data/trigrams/hu PRE-CREATION 
  data/trigrams/id PRE-CREATION 
  data/trigrams/is PRE-CREATION 
  data/trigrams/it PRE-CREATION 
  data/parsetrigrams.cpp PRE-CREATION 
  data/trigrams/af PRE-CREATION 
  data/trigrams/ar PRE-CREATION 
  data/trigrams/az PRE-CREATION 
  data/trigrams/bg PRE-CREATION 
  data/trigrams/ca PRE-CREATION 
  data/trigrams/cs PRE-CREATION 
  data/trigrams/cy PRE-CREATION 
  data/trigrams/da PRE-CREATION 
  data/trigrams/de PRE-CREATION 
  data/trigrams/en PRE-CREATION 
  data/trigrams/es PRE-CREATION 
  data/trigrams/et PRE-CREATION 
  data/trigrams/eu PRE-CREATION 
  data/trigrams/fa PRE-CREATION 
  data/trigrams/fi PRE-CREATION 
  CMakeLists.txt 1fdcf1e 
  README.md 63e2c6a 
  autotests/CMakeLists.txt e9fc573 
  data/CMakeLists.txt PRE-CREATION 

Diff: https://git.reviewboard.kde.org/r/114717/diff/

Testing
-------

mostly using test_highlighter.

Thanks,

Martin Tobias Holmedahl Sandsmark

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kde.org/pipermail/kde-core-devel/attachments/20140108/3dc78a94/attachment.htm>